Chinese_News_article_gpt_llm_trainer_v2,_with_GPT_3_5_Fine_Tuning.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ncellepola/gpt-llm-trainer/blob/main/Chinese_News_article_gpt_llm_trainer_v2%2C_with_GPT_3_5_Fine_Tuning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Describe your model -> fine-tuned GPT-3.5\n",
        "By Matt Shumer (https://twitter.com/mattshumer_)\n",
        "\n",
        "The goal of this notebook is to experiment with a new way to make it very easy to build a task-specific model for your use-case.\n",
        "\n",
        "First, use the best GPU available (go to Runtime -> change runtime type)\n",
        "\n",
        "To create your model, just go to the first code cell, and describe the model you want to build in the prompt. Be descriptive and clear.\n",
        "\n",
        "Select a temperature (high=creative, low=precise), and the number of training examples to generate to train the model. From there, just run all the cells.\n",
        "\n",
        "You can change the model you want to fine-tune by changing `model_name` in the `Define Hyperparameters` cell."
      ],
      "metadata": {
        "id": "wM8MRkf8Dr94"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Data generation step"
      ],
      "metadata": {
        "id": "Way3_PuPpIuE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Write your prompt here. Make it as descriptive as possible!\n",
        "\n",
        "Then, choose the temperature (between 0 and 1) to use when generating data. Lower values are great for precise tasks, like writing code, whereas larger values are better for creative tasks, like writing stories.\n",
        "\n",
        "Finally, choose how many examples you want to generate. The more you generate, a) the longer it takes and b) the more expensive data generation will be. But generally, more examples will lead to a higher-quality model. 100 is usually the minimum to start."
      ],
      "metadata": {
        "id": "lY-3DvlIpVSl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "prompt = \"A model that takes in Chinese informative news articles , and responds with a well-understood, condensed summary of the article in English language.\"\n",
        "temperature = .4\n",
        "number_of_examples = 50"
      ],
      "metadata": {
        "id": "R7WKZyxtpUPS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Run this to generate the dataset."
      ],
      "metadata": {
        "id": "1snNou5PrIci"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install openai tenacity"
      ],
      "metadata": {
        "id": "zuL2UaqlsmBD",
        "outputId": "38f24f48-ea60-4af9-d52f-30cb043a85fe",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting openai\n",
            "  Downloading openai-1.2.3-py3-none-any.whl (220 kB)\n",
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/220.3 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m \u001b[32m215.0/220.3 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m220.3/220.3 kB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tenacity in /usr/local/lib/python3.10/dist-packages (8.2.3)\n",
            "Requirement already satisfied: anyio<4,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai) (3.7.1)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai) (1.7.0)\n",
            "Collecting httpx<1,>=0.23.0 (from openai)\n",
            "  Downloading httpx-0.25.1-py3-none-any.whl (75 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.0/75.0 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai) (1.10.13)\n",
            "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai) (4.66.1)\n",
            "Requirement already satisfied: typing-extensions<5,>=4.5 in /usr/local/lib/python3.10/dist-packages (from openai) (4.5.0)\n",
            "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<4,>=3.5.0->openai) (3.4)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<4,>=3.5.0->openai) (1.3.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<4,>=3.5.0->openai) (1.1.3)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai) (2023.7.22)\n",
            "Collecting httpcore (from httpx<1,>=0.23.0->openai)\n",
            "  Downloading httpcore-1.0.2-py3-none-any.whl (76 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.9/76.9 kB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting h11<0.15,>=0.13 (from httpcore->httpx<1,>=0.23.0->openai)\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m9.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: h11, httpcore, httpx, openai\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "llmx 0.0.15a0 requires tiktoken, which is not installed.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed h11-0.14.0 httpcore-1.0.2 httpx-0.25.1 openai-1.2.3\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install openai==0.28"
      ],
      "metadata": {
        "id": "Xz94B6vWHgFO",
        "outputId": "7e2ee6c1-c321-476f-d891-93a378764a62",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 567
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting openai==0.28\n",
            "  Downloading openai-0.28.0-py3-none-any.whl (76 kB)\n",
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/76.5 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━\u001b[0m \u001b[32m71.7/76.5 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.5/76.5 kB\u001b[0m \u001b[31m1.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: requests>=2.20 in /usr/local/lib/python3.10/dist-packages (from openai==0.28) (2.31.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai==0.28) (4.66.1)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from openai==0.28) (3.8.6)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (2023.7.22)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (23.1.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (4.0.3)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (1.4.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (1.3.1)\n",
            "Installing collected packages: openai\n",
            "  Attempting uninstall: openai\n",
            "    Found existing installation: openai 1.2.3\n",
            "    Uninstalling openai-1.2.3:\n",
            "      Successfully uninstalled openai-1.2.3\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "llmx 0.0.15a0 requires tiktoken, which is not installed.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed openai-0.28.0\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "openai"
                ]
              }
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "import openai\n",
        "import random\n",
        "from tenacity import retry, stop_after_attempt, wait_exponential\n",
        "\n",
        "openai.api_key = \"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\"\n",
        "\n",
        "N_RETRIES = 3\n",
        "\n",
        "@retry(stop=stop_after_attempt(N_RETRIES), wait=wait_exponential(multiplier=1, min=4, max=70))\n",
        "def generate_example(prompt, prev_examples, temperature=.5):\n",
        "    messages=[\n",
        "        {\n",
        "            \"role\": \"system\",\n",
        "            \"content\": f\"You are generating data which will be used to train a machine learning model.\\n\\nYou will be given a high-level description of the model we want to train, and from that, you will generate data samples, each with a prompt/response pair.\\n\\nYou will do so in this format:\\n```\\nprompt\\n-----------\\n$prompt_goes_here\\n-----------\\n\\nresponse\\n-----------\\n$response_goes_here\\n-----------\\n```\\n\\nOnly one prompt/response pair should be generated per turn.\\n\\nFor each turn, make the example slightly more complex than the last, while ensuring diversity.\\n\\nMake sure your samples are unique and diverse, yet high-quality and complex enough to train a well-performing model.\\n\\nHere is the type of model we want to train:\\n`{prompt}`\"\n",
        "        }\n",
        "    ]\n",
        "\n",
        "    if len(prev_examples) > 0:\n",
        "        if len(prev_examples) > 8:\n",
        "            prev_examples = random.sample(prev_examples, 8)\n",
        "        for example in prev_examples:\n",
        "            messages.append({\n",
        "                \"role\": \"assistant\",\n",
        "                \"content\": example\n",
        "            })\n",
        "\n",
        "    response = openai.ChatCompletion.create(\n",
        "        model=\"gpt-4\",\n",
        "        messages=messages,\n",
        "        temperature=temperature,\n",
        "        max_tokens=1000,\n",
        "    )\n",
        "\n",
        "    return response.choices[0].message['content']\n",
        "\n",
        "# Generate examples\n",
        "prev_examples = []\n",
        "for i in range(number_of_examples):\n",
        "    print(f'Generating example {i}')\n",
        "    example = generate_example(prompt, prev_examples, temperature)\n",
        "    prev_examples.append(example)\n",
        "\n",
        "print(prev_examples)"
      ],
      "metadata": {
        "id": "Rdsd82ngpHCG",
        "outputId": "1827f3fe-605d-4937-d1b1-f2b84df7c8ed",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Generating example 0\n",
            "Generating example 1\n",
            "Generating example 2\n",
            "Generating example 3\n",
            "Generating example 4\n",
            "Generating example 5\n",
            "Generating example 6\n",
            "Generating example 7\n",
            "Generating example 8\n",
            "Generating example 9\n",
            "Generating example 10\n",
            "Generating example 11\n",
            "Generating example 12\n",
            "Generating example 13\n",
            "Generating example 14\n",
            "Generating example 15\n",
            "Generating example 16\n",
            "Generating example 17\n",
            "Generating example 18\n",
            "Generating example 19\n",
            "Generating example 20\n",
            "Generating example 21\n",
            "Generating example 22\n",
            "Generating example 23\n",
            "Generating example 24\n",
            "Generating example 25\n",
            "Generating example 26\n",
            "Generating example 27\n",
            "Generating example 28\n",
            "Generating example 29\n",
            "Generating example 30\n",
            "Generating example 31\n",
            "Generating example 32\n",
            "Generating example 33\n",
            "Generating example 34\n",
            "Generating example 35\n",
            "Generating example 36\n",
            "Generating example 37\n",
            "Generating example 38\n",
            "Generating example 39\n",
            "Generating example 40\n",
            "Generating example 41\n",
            "Generating example 42\n",
            "Generating example 43\n",
            "Generating example 44\n",
            "Generating example 45\n",
            "Generating example 46\n",
            "Generating example 47\n",
            "Generating example 48\n",
            "Generating example 49\n",
            "[\"prompt\\n-----------\\n本周，中国政府宣布了一项新的经济政策，旨在刺激国内消费和投资。这项政策包括降低关税，提高政府支出，以及推动科技创新。政府表示，这项政策将有助于促进经济增长，提高就业率，以及提高人民的生活水平。此外，政府还表示，这项政策将有助于中国在全球经济中的地位。\\n-----------\\nresponse\\n-----------\\nThis week, the Chinese government announced a new economic policy aimed at stimulating domestic consumption and investment. The policy includes lowering tariffs, increasing government spending, and promoting technological innovation. The government stated that this policy will help promote economic growth, increase employment rates, and improve people's living standards. Additionally, the government said that this policy will aid China's position in the global economy.\\n\", \"prompt\\n-----------\\n中国科学家们在人工智能领域取得了重大突破。他们开发出一种新型的深度学习算法，能够在图像识别和语音识别等任务上表现出色。这项技术的开发不仅将推动中国的科技创新，也可能对全球的人工智能研究产生深远影响。中国政府表示，将会全力支持这项技术的进一步研发和应用。\\n-----------\\nresponse\\n-----------\\nChinese scientists have made significant breakthroughs in the field of artificial intelligence. They have developed a new type of deep learning algorithm that excels in tasks such as image recognition and speech recognition. The development of this technology will not only promote China's technological innovation, but may also have a profound impact on global artificial intelligence research. The Chinese government stated that it will fully support the further research and application of this technology.\\n\", \"prompt\\n-----------\\n中国最新的环保政策已经开始见效。政府最近发布的数据显示，中国的空气质量在过去的一年中有了显著的改善。这是自中国政府几年前开始实施严格的环保政策以来的一个重要成果。政府表示，将继续努力，以实现更大的环保目标。\\n-----------\\nresponse\\n-----------\\nChina's latest environmental policy has begun to take effect. Recent data released by the government shows a significant improvement in China's air quality over the past year. This is an important achievement since the Chinese government began implementing strict environmental policies a few years ago. The government stated that it will continue to strive to achieve greater environmental goals.\\n\", 'prompt\\n-----------\\n中国政府近日宣布，将投入大量资金用于教育改革。这项改革的目标是提高教育质量，以满足经济发展的需要。政府计划在未来五年内，将教育支出的比例提高到GDP的4%。此外，政府还计划增加对农村和贫困地区的教育投入，以缩小城乡教育差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in education reform. The goal of the reform is to improve the quality of education to meet the needs of economic development. The government plans to increase the proportion of education expenditure to 4% of GDP in the next five years. In addition, the government also plans to increase education investment in rural and impoverished areas to narrow the gap in education between urban and rural areas.\\n', \"prompt\\n-----------\\n中国的科技巨头阿里巴巴集团宣布，将在未来五年内投入1000亿人民币用于发展云计算技术。这项投资将用于研发新的云计算产品，提升数据中心的能效，以及培训更多的云计算人才。阿里巴巴表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球云计算市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Alibaba Group announced that it will invest 100 billion yuan in the development of cloud computing technology over the next five years. This investment will be used for the research and development of new cloud computing products, improving the energy efficiency of data centers, and training more cloud computing talents. Alibaba stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global cloud computing market.\\n\", \"prompt\\n-----------\\n中国政府近日宣布，将大力推动绿色能源的发展，以减少碳排放并应对全球气候变化。政府计划在未来十年内，将可再生能源的份额提高到能源总消费的20%。此外，政府还将提供一系列的财政和税收优惠，以鼓励企业和个人使用绿色能源。这项政策被视为中国应对气候变化的重要步骤。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will vigorously promote the development of green energy to reduce carbon emissions and address global climate change. The government plans to increase the share of renewable energy to 20% of total energy consumption in the next decade. In addition, the government will provide a series of fiscal and tax incentives to encourage companies and individuals to use green energy. This policy is seen as an important step in China's response to climate change.\", \"prompt\\n-----------\\n中国的电商巨头京东集团宣布，将在未来三年内投资200亿人民币用于推动物流自动化。这项投资将用于研发和应用新的自动化技术，包括无人驾驶货车、无人机和机器人。京东表示，这项投资将有助于提高物流效率，降低运营成本，同时也将推动中国的科技创新。\\n-----------\\nresponse\\n-----------\\nChina's e-commerce giant JD.com announced that it will invest 20 billion yuan over the next three years to promote logistics automation. This investment will be used for the research and application of new automation technologies, including self-driving trucks, drones, and robots. JD.com stated that this investment will help improve logistics efficiency, reduce operating costs, and also promote technological innovation in China.\\n\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于公共卫生体系的改革。这项改革的目标是提高公共卫生服务的质量和可及性，以应对未来可能出现的公共卫生危机。政府计划在未来五年内，将公共卫生支出的比例提高到GDP的3%。此外，政府还计划增加对农村和贫困地区的公共卫生投入，以缩小城乡公共卫生差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in the reform of the public health system over the next five years. The goal of the reform is to improve the quality and accessibility of public health services to cope with potential public health crises in the future. The government plans to increase the proportion of public health expenditure to 3% of GDP in the next five years. In addition, the government also plans to increase public health investment in rural and impoverished areas to narrow the gap in public health between urban and rural areas.', \"prompt\\n-----------\\n中国的科技巨头腾讯公司宣布，将在未来五年内投入500亿人民币用于发展区块链技术。这项投资将用于研发新的区块链产品，提升区块链的安全性和效率，以及培训更多的区块链人才。腾讯表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球区块链市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Tencent announced that it will invest 50 billion yuan in the development of blockchain technology over the next five years. This investment will be used for the research and development of new blockchain products, improving the security and efficiency of blockchain, and training more blockchain talents. Tencent stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global blockchain market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于基础设施建设。这项计划的目标是改善交通网络，提高公共服务设施的质量，以及推动经济发展。政府计划在未来五年内，将基础设施投资的比例提高到GDP的5%。此外，政府还计划增加对农村和贫困地区的基础设施投入，以缩小城乡基础设施差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in infrastructure construction over the next five years. The goal of this plan is to improve the transportation network, improve the quality of public service facilities, and promote economic development. The government plans to increase the proportion of infrastructure investment to 5% of GDP in the next five years. In addition, the government also plans to increase infrastructure investment in rural and impoverished areas to narrow the gap in infrastructure between urban and rural areas.', \"prompt\\n-----------\\n中国的电动汽车市场正在快速增长。根据最近的数据，中国已经成为全球最大的电动汽车市场，销售量超过了美国和欧洲。中国政府的政策支持，以及消费者对环保和新技术的接受度，都有助于推动这个市场的发展。预计在未来几年内，中国的电动汽车市场将继续保持强劲的增长势头。\\n-----------\\nresponse\\n-----------\\nChina's electric vehicle market is growing rapidly. According to recent data, China has become the world's largest electric vehicle market, with sales surpassing those in the US and Europe. The Chinese government's policy support and consumers' acceptance of environmental protection and new technologies have helped drive the development of this market. It is expected that China's electric vehicle market will continue to maintain strong growth momentum in the coming years.\", \"prompt\\n-----------\\n中国政府最近宣布了一项新的旅游政策，旨在吸引更多的国际游客。这项政策包括放宽签证政策，提高旅游设施的质量，以及推广中国的文化和历史。政府表示，这项政策将有助于推动旅游业的发展，增加就业机会，以及提高国际游客对中国的认知。预计在未来几年内，中国的国际旅游市场将继续保持强劲的增长势头。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced a new tourism policy aimed at attracting more international tourists. The policy includes relaxing visa policies, improving the quality of tourism facilities, and promoting China's culture and history. The government stated that this policy will help promote the development of the tourism industry, increase employment opportunities, and enhance international tourists' awareness of China. It is expected that China's international tourism market will continue to maintain strong growth momentum in the coming years.\", \"prompt\\n-----------\\n中国的科技巨头腾讯公司宣布，将在未来三年内投入500亿人民币用于发展人工智能和云计算技术。这项投资将用于研发新的人工智能产品，提升数据中心的能效，以及培训更多的人工智能和云计算人才。腾讯表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球人工智能和云计算市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Tencent announced that it will invest 50 billion yuan in the development of artificial intelligence and cloud computing technology over the next three years. This investment will be used for the research and development of new artificial intelligence products, improving the energy efficiency of data centers, and training more artificial intelligence and cloud computing talents. Tencent stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global artificial intelligence and cloud computing market.\", \"prompt\\n-----------\\n中国最新的数据显示，国内的电子商务市场在过去的一年中持续增长。这主要得益于中国消费者对在线购物的热爱，以及中国政府对电子商务行业的大力支持。预计在未来几年内，中国的电子商务市场将继续保持强劲的增长势头。这将为中国的经济发展提供强大的动力。\\n-----------\\nresponse\\n-----------\\nChina's latest data shows that the domestic e-commerce market has continued to grow over the past year. This is mainly due to Chinese consumers' love for online shopping and the strong support of the Chinese government for the e-commerce industry. It is expected that China's e-commerce market will continue to maintain strong growth momentum in the coming years. This will provide a strong impetus for China's economic development.\", \"prompt\\n-----------\\n中国的科技巨头华为公司宣布，将在未来五年内投入1000亿人民币用于研发5G技术。这项投资将用于研发新的5G产品，提升5G网络的覆盖率和稳定性，以及培训更多的5G技术人才。华为表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球5G市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Huawei announced that it will invest 100 billion yuan in the research and development of 5G technology over the next five years. This investment will be used for the research and development of new 5G products, improving the coverage and stability of the 5G network, and training more 5G technology talents. Huawei stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global 5G market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于环境保护。这项计划的目标是改善空气质量，保护水资源，以及减少碳排放。政府计划在未来五年内，将环保投资的比例提高到GDP的3%。此外，政府还计划增加对农村和贫困地区的环保投入，以缩小城乡环保差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in environmental protection over the next five years. The goal of this plan is to improve air quality, protect water resources, and reduce carbon emissions. The government plans to increase the proportion of environmental protection investment to 3% of GDP in the next five years. In addition, the government also plans to increase environmental protection investment in rural and impoverished areas to narrow the gap in environmental protection between urban and rural areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于环保项目。这项计划的目标是提高环境质量，减少污染，以及推动绿色经济发展。政府计划在未来五年内，将环保投资的比例提高到GDP的3%。此外，政府还计划增加对农村和贫困地区的环保投入，以改善这些地区的环境质量。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in environmental protection projects over the next five years. The goal of this plan is to improve environmental quality, reduce pollution, and promote the development of the green economy. The government plans to increase the proportion of environmental protection investment to 3% of GDP in the next five years. In addition, the government also plans to increase environmental protection investment in rural and impoverished areas to improve the environmental quality of these areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于教育改革。这项计划的目标是提高教育质量，提升教师的专业能力，以及增加对贫困地区的教育投入。政府计划在未来五年内，将教育投资的比例提高到GDP的4%。此外，政府还计划增加对农村和贫困地区的教育投入，以缩小城乡教育差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in education reform over the next five years. The goal of this plan is to improve the quality of education, enhance the professional abilities of teachers, and increase education investment in impoverished areas. The government plans to increase the proportion of education investment to 4% of GDP in the next five years. In addition, the government also plans to increase education investment in rural and impoverished areas to narrow the gap in education between urban and rural areas.', \"prompt\\n-----------\\n中国的科技巨头华为公司宣布，将在未来五年内投入200亿美元用于发展5G网络技术。这项投资将用于研发新的5G产品，提升网络的速度和稳定性，以及培训更多的5G网络技术人才。华为表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球5G网络市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Huawei announced that it will invest 20 billion dollars in the development of 5G network technology over the next five years. This investment will be used for the research and development of new 5G products, improving the speed and stability of the network, and training more 5G network technology talents. Huawei stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global 5G network market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于医疗改革。这项改革的目标是提高医疗服务质量，以满足人民日益增长的健康需求。政府计划在未来五年内，将医疗支出的比例提高到GDP的6%。此外，政府还计划增加对农村和贫困地区的医疗投入，以改善这些地区的医疗服务质量。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in healthcare reform over the next five years. The goal of the reform is to improve the quality of healthcare services to meet the growing health needs of the people. The government plans to increase the proportion of healthcare expenditure to 6% of GDP in the next five years. In addition, the government also plans to increase healthcare investment in rural and impoverished areas to improve the quality of healthcare services in these areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于医疗改革。这项改革的目标是提高医疗服务质量，以满足人民的健康需求。政府计划在未来五年内，将医疗支出的比例提高到GDP的5%。此外，政府还计划增加对农村和贫困地区的医疗投入，以缩小城乡医疗差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in healthcare reform over the next five years. The goal of the reform is to improve the quality of healthcare services to meet the health needs of the people. The government plans to increase the proportion of healthcare expenditure to 5% of GDP in the next five years. In addition, the government also plans to increase healthcare investment in rural and impoverished areas to narrow the gap in healthcare between urban and rural areas.', \"prompt\\n-----------\\n中国的科技巨头阿里巴巴集团宣布，将在未来五年内投入1000亿人民币用于发展云计算技术。这项投资将用于研发新的云计算产品，提升数据中心的能效，以及培训更多的云计算人才。阿里巴巴表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球云计算市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Alibaba Group announced that it will invest 100 billion yuan in the development of cloud computing technology over the next five years. This investment will be used for the research and development of new cloud computing products, improving the energy efficiency of data centers, and training more cloud computing talents. Alibaba stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global cloud computing market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于教育改革。这项计划的目标是提高教育质量，减少教育不平等，以及推动教育创新。政府计划在未来五年内，将教育投资的比例提高到GDP的4%。此外，政府还计划增加对农村和贫困地区的教育投入，以改善这些地区的教育质量。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in educational reform over the next five years. The goal of this plan is to improve the quality of education, reduce educational inequality, and promote educational innovation. The government plans to increase the proportion of education investment to 4% of GDP in the next five years. In addition, the government also plans to increase education investment in rural and impoverished areas to improve the quality of education in these areas.', \"prompt\\n-----------\\n中国的科技巨头阿里巴巴集团宣布，将在未来五年内投入500亿人民币用于研发人工智能技术。这项投资将用于研发新的人工智能产品，提升人工智能技术的应用和稳定性，以及培训更多的人工智能技术人才。阿里巴巴表示，这项投资将有助于推动中国的科技创新，同时也将提高中国在全球人工智能市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Alibaba Group announced that it will invest 50 billion yuan in the research and development of artificial intelligence technology over the next five years. This investment will be used for the research and development of new artificial intelligence products, improving the application and stability of artificial intelligence technology, and training more artificial intelligence technology talents. Alibaba stated that this investment will help promote technological innovation in China and also enhance China's competitiveness in the global artificial intelligence market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于教育改革。这项改革的目标是提高教育质量，以满足人民的学习需求。政府计划在未来五年内，将教育支出的比例提高到GDP的4%。此外，政府还计划增加对农村和贫困地区的教育投入，以缩小城乡教育差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in education reform over the next five years. The goal of the reform is to improve the quality of education to meet the learning needs of the people. The government plans to increase the proportion of education expenditure to 4% of GDP in the next five years. In addition, the government also plans to increase education investment in rural and impoverished areas to narrow the gap in education between urban and rural areas.', \"prompt\\n-----------\\n中国的科技巨头腾讯公司宣布，将在未来五年内投入500亿人民币用于发展人工智能技术。这项投资将用于研发新的人工智能产品，提升算法的精度和效率，以及培训更多的人工智能人才。腾讯表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球人工智能市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Tencent announced that it will invest 50 billion yuan in the development of artificial intelligence technology over the next five years. This investment will be used for the research and development of new AI products, improving the accuracy and efficiency of algorithms, and training more AI talents. Tencent stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global AI market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于医疗改革。这项改革的目标是提高医疗服务质量，以满足人民日益增长的健康需求。政府计划在未来五年内，将医疗支出的比例提高到GDP的7%。此外，政府还计划增加对农村和贫困地区的医疗投入，以缩小城乡医疗差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in medical reform over the next five years. The goal of this reform is to improve the quality of medical services to meet the growing health needs of the people. The government plans to increase the proportion of medical expenditure to 7% of GDP in the next five years. In addition, the government also plans to increase medical investment in rural and impoverished areas to narrow the gap in medical care between urban and rural areas.', \"prompt\\n-----------\\n中国政府宣布，将在未来五年内投入1000亿人民币用于发展5G网络。这项投资将用于建设新的5G基站，提升网络覆盖率，以及培训更多的5G网络人才。政府表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球5G网络市场的竞争力。\\n-----------\\nresponse\\n-----------\\nThe Chinese government announced that it will invest 100 billion yuan in the development of 5G networks over the next five years. This investment will be used to build new 5G base stations, improve network coverage, and train more 5G network talents. The government stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global 5G network market.\", \"prompt\\n-----------\\n中国政府近日发布了一项新的政策，旨在推动国内的绿色建筑发展。根据这项政策，政府将提供一系列的财政和税收优惠，以鼓励企业和个人投资绿色建筑。政府表示，这项政策将有助于提高中国的能源效率，减少碳排放，同时也将推动中国的绿色经济发展。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently issued a new policy aimed at promoting the development of green buildings in the country. According to this policy, the government will provide a series of fiscal and tax incentives to encourage companies and individuals to invest in green buildings. The government stated that this policy will help improve China's energy efficiency, reduce carbon emissions, and also promote the development of China's green economy.\", \"prompt\\n-----------\\n中国的科技巨头阿里巴巴集团宣布，将在未来五年内投入1000亿人民币用于推动人工智能和云计算技术的发展。这项投资将用于研发新的人工智能和云计算产品，提升数据中心的能效，以及培训更多的人工智能和云计算人才。阿里巴巴表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球人工智能和云计算市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Alibaba Group announced that it will invest 100 billion yuan in the development of artificial intelligence and cloud computing technology over the next five years. This investment will be used for the research and development of new artificial intelligence and cloud computing products, improving the energy efficiency of data centers, and training more artificial intelligence and cloud computing talents. Alibaba stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global artificial intelligence and cloud computing market.\", \"prompt\\n-----------\\n中国的科技巨头腾讯公司宣布，将在未来五年内投入500亿人民币用于研发区块链技术。这项投资将用于研发新的区块链产品，提升区块链技术的应用和稳定性，以及培训更多的区块链技术人才。腾讯表示，这项投资将有助于推动中国的科技创新，同时也将提高中国在全球区块链市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Tencent announced that it will invest 50 billion yuan in the research and development of blockchain technology over the next five years. This investment will be used for the research and development of new blockchain products, improving the application and stability of blockchain technology, and training more blockchain technology talents. Tencent stated that this investment will help promote technological innovation in China and also enhance China's competitiveness in the global blockchain market.\", \"prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于环境保护和改善。这项改革的目标是提高环境质量，以满足人民对美好生活环境的期待。政府计划在未来五年内，将环保支出的比例提高到GDP的3%。此外，政府还计划增加对农村和贫困地区的环保投入，以缩小城乡环境质量差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in environmental protection and improvement over the next five years. The goal of the reform is to improve the quality of the environment to meet people's expectations for a better living environment. The government plans to increase the proportion of environmental protection expenditure to 3% of GDP in the next five years. In addition, the government also plans to increase environmental protection investment in rural and impoverished areas to narrow the gap in environmental quality between urban and rural areas.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于科技创新。这项计划的目标是推动科技进步，培养科技人才，以及推动经济发展。政府计划在未来五年内，将科技投资的比例提高到GDP的5%。此外，政府还计划增加对农村和贫困地区的科技投入，以改善这些地区的科技发展水平。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in technological innovation over the next five years. The goal of this plan is to promote technological progress, cultivate technological talents, and promote economic development. The government plans to increase the proportion of technological investment to 5% of GDP in the next five years. In addition, the government also plans to increase technological investment in rural and impoverished areas to improve the level of technological development in these areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于环保项目。这项计划的目标是提高环保设施的质量，减少污染，以及推动绿色经济的发展。政府计划在未来五年内，将环保投资的比例提高到GDP的5%。此外，政府还计划增加对农村和贫困地区的环保投入，以改善这些地区的环境质量。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in environmental protection projects over the next five years. The goal of this plan is to improve the quality of environmental protection facilities, reduce pollution, and promote the development of the green economy. The government plans to increase the proportion of environmental protection investment to 5% of GDP in the next five years. In addition, the government also plans to increase environmental protection investment in rural and impoverished areas to improve the environmental quality of these areas.', \"prompt\\n-----------\\n中国的科技巨头腾讯公司宣布，将在未来五年内投入500亿人民币用于推动区块链技术的发展。这项投资将用于研发新的区块链产品，提升数据安全，以及培训更多的区块链人才。腾讯表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球区块链市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Tencent announced that it will invest 50 billion yuan in the development of blockchain technology over the next five years. This investment will be used for the research and development of new blockchain products, improving data security, and training more blockchain talents. Tencent stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global blockchain market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于医疗改革。这项改革的目标是提高医疗服务质量，以满足人民的健康需求。政府计划在未来五年内，将医疗支出的比例提高到GDP的7%。此外，政府还计划增加对农村和贫困地区的医疗投入，以缩小城乡医疗差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in healthcare reform over the next five years. The goal of the reform is to improve the quality of medical services to meet the health needs of the people. The government plans to increase the proportion of healthcare expenditure to 7% of GDP in the next five years. In addition, the government also plans to increase healthcare investment in rural and impoverished areas to narrow the gap in healthcare between urban and rural areas.', \"prompt\\n-----------\\n中国的科技巨头华为公司宣布，将在未来五年内投入500亿人民币用于发展5G技术。这项投资将用于研发新的5G产品，提升5G网络的覆盖率和质量，以及培训更多的5G技术人才。华为表示，这项投资将有助于推动中国的数字经济发展，同时也将提高中国在全球5G市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Huawei announced that it will invest 50 billion yuan in the development of 5G technology over the next five years. This investment will be used for the research and development of new 5G products, improving the coverage and quality of 5G networks, and training more 5G technology talents. Huawei stated that this investment will help promote the development of China's digital economy and also enhance China's competitiveness in the global 5G market.\", \"prompt\\n-----------\\n中国的科技巨头华为公司宣布，将在未来五年内投入500亿人民币用于研发5G技术。这项投资将用于研发新的5G产品，提升5G技术的应用和稳定性，以及培训更多的5G技术人才。华为表示，这项投资将有助于推动中国的科技创新，同时也将提高中国在全球5G市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Huawei announced that it will invest 50 billion yuan in the research and development of 5G technology over the next five years. This investment will be used for the research and development of new 5G products, improving the application and stability of 5G technology, and training more 5G technology talents. Huawei stated that this investment will help promote technological innovation in China and also enhance China's competitiveness in the global 5G market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于公共卫生体系的改革。这项改革的目标是提高公共卫生服务的质量和效率，以满足人民的健康需求。政府计划在未来五年内，将公共卫生支出的比例提高到GDP的6%。此外，政府还计划增加对农村和贫困地区的公共卫生投入，以缩小城乡公共卫生服务的差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in public health system reform over the next five years. The goal of the reform is to improve the quality and efficiency of public health services to meet the health needs of the people. The government plans to increase the proportion of public health expenditure to 6% of GDP in the next five years. In addition, the government also plans to increase public health investment in rural and impoverished areas to narrow the gap in public health services between urban and rural areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于医疗改革。这项计划的目标是提高医疗服务质量，提升医生的专业能力，以及增加对贫困地区的医疗投入。政府计划在未来五年内，将医疗投资的比例提高到GDP的5%。此外，政府还计划增加对农村和贫困地区的医疗投入，以缩小城乡医疗差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in medical reform over the next five years. The goal of this plan is to improve the quality of medical services, enhance the professional abilities of doctors, and increase medical investment in impoverished areas. The government plans to increase the proportion of medical investment to 5% of GDP in the next five years. In addition, the government also plans to increase medical investment in rural and impoverished areas to narrow the gap in medical services between urban and rural areas.', \"prompt\\n-----------\\n中国的科技巨头腾讯公司宣布，将在未来五年内投入500亿人民币用于发展区块链技术。这项投资将用于研发新的区块链产品，提升区块链技术的应用和稳定性，以及培训更多的区块链技术人才。腾讯表示，这项投资将有助于推动中国的科技创新，同时也将提高中国在全球区块链市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Tencent announced that it will invest 50 billion yuan in the development of blockchain technology over the next five years. This investment will be used for the research and development of new blockchain products, improving the application and stability of blockchain technology, and training more blockchain technology talents. Tencent stated that this investment will help promote technological innovation in China and also enhance China's competitiveness in the global blockchain market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于发展新能源汽车。这项计划的目标是推动新能源汽车的研发，提高新能源汽车的质量和性能，以及推动绿色经济的发展。政府计划在未来五年内，将新能源汽车投资的比例提高到GDP的5%。此外，政府还计划增加对农村和贫困地区的新能源汽车投入，以改善这些地区的交通状况。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in the development of new energy vehicles over the next five years. The goal of this plan is to promote the research and development of new energy vehicles, improve the quality and performance of new energy vehicles, and promote the development of the green economy. The government plans to increase the proportion of new energy vehicle investment to 5% of GDP in the next five years. In addition, the government also plans to increase new energy vehicle investment in rural and impoverished areas to improve the transportation conditions in these areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于环境保护和可持续发展。这项改革的目标是提高环境质量，以满足人民的生活需求。政府计划在未来五年内，将环保支出的比例提高到GDP的3%。此外，政府还计划增加对农村和贫困地区的环保投入，以缩小城乡环保差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in environmental protection and sustainable development over the next five years. The goal of the reform is to improve environmental quality to meet the living needs of the people. The government plans to increase the proportion of environmental protection expenditure to 3% of GDP in the next five years. In addition, the government also plans to increase environmental protection investment in rural and impoverished areas to narrow the gap in environmental protection between urban and rural areas.', \"prompt\\n-----------\\n中国政府最近宣布了一项新的环保政策，旨在减少空气污染。这项政策包括限制化石燃料的使用，提高可再生能源的使用比例，以及加强环保法规的执行。政府表示，这项政策将有助于改善空气质量，保护公众健康，以及应对气候变化。预计在未来几年内，中国的空气质量将有所改善。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced a new environmental policy aimed at reducing air pollution. The policy includes limiting the use of fossil fuels, increasing the proportion of renewable energy use, and strengthening the enforcement of environmental regulations. The government stated that this policy will help improve air quality, protect public health, and address climate change. It is expected that China's air quality will improve in the coming years.\", \"prompt\\n-----------\\n中国的科技巨头百度公司宣布，将在未来五年内投入500亿人民币用于发展人工智能技术。这项投资将用于研发新的人工智能产品，提升人工智能技术的应用和稳定性，以及培训更多的人工智能技术人才。百度表示，这项投资将有助于推动中国的科技创新，同时也将提高中国在全球人工智能市场的竞争力。\\n-----------\\nresponse\\n-----------\\nChina's tech giant Baidu announced that it will invest 50 billion yuan in the development of artificial intelligence technology over the next five years. This investment will be used for the research and development of new artificial intelligence products, improving the application and stability of artificial intelligence technology, and training more artificial intelligence technology talents. Baidu stated that this investment will help promote technological innovation in China and also enhance China's competitiveness in the global artificial intelligence market.\", 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于教育改革。这项改革的目标是提高教育质量，以满足人民的学习需求。政府计划在未来五年内，将教育支出的比例提高到GDP的4%。此外，政府还计划增加对农村和贫困地区的教育投入，以缩小城乡教育差距。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in education reform over the next five years. The goal of this reform is to improve the quality of education to meet the learning needs of the people. The government plans to increase the proportion of education expenditure to 4% of GDP in the next five years. In addition, the government also plans to increase education investment in rural and impoverished areas to narrow the gap in education between urban and rural areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于教育改革。这项计划的目标是提高教育质量，减少教育不平等，以及推动创新教育的发展。政府计划在未来五年内，将教育投资的比例提高到GDP的4%。此外，政府还计划增加对农村和贫困地区的教育投入，以改善这些地区的教育质量。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in educational reform over the next five years. The goal of this plan is to improve the quality of education, reduce educational inequality, and promote the development of innovative education. The government plans to increase the proportion of educational investment to 4% of GDP in the next five years. In addition, the government also plans to increase educational investment in rural and impoverished areas to improve the quality of education in these areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于基础设施建设。这项计划的目标是提高基础设施的质量和覆盖率，以满足人民的日常需求。政府计划在未来五年内，将基础设施投资的比例提高到GDP的5%。此外，政府还计划增加对农村和贫困地区的基础设施投入，以改善这些地区的生活条件。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in infrastructure construction over the next five years. The goal of this plan is to improve the quality and coverage of infrastructure to meet the daily needs of the people. The government plans to increase the proportion of infrastructure investment to 5% of GDP in the next five years. In addition, the government also plans to increase infrastructure investment in rural and impoverished areas to improve living conditions in these areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于公共卫生系统的改革。这项改革的目标是提高公共卫生服务的质量和效率，以满足人民的健康需求。政府计划在未来五年内，将公共卫生支出的比例提高到GDP的7%。此外，政府还计划增加对农村和贫困地区的公共卫生投入，以改善这些地区的卫生状况。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in the reform of the public health system over the next five years. The goal of the reform is to improve the quality and efficiency of public health services to meet the health needs of the people. The government plans to increase the proportion of public health expenditure to 7% of GDP in the next five years. In addition, the government also plans to increase public health investment in rural and impoverished areas to improve the sanitation conditions in these areas.', 'prompt\\n-----------\\n中国政府近日宣布，将在未来五年内投入大量资金用于科技创新。这项计划的目标是推动科技创新，提升科研能力，以及培养更多的科技人才。政府计划在未来五年内，将科技投资的比例提高到GDP的2.5%。此外，政府还计划增加对农村和贫困地区的科技投入，以改善这些地区的科技水平。\\n-----------\\nresponse\\n-----------\\nThe Chinese government recently announced that it will invest a significant amount of money in technological innovation over the next five years. The goal of this plan is to promote technological innovation, enhance scientific research capabilities, and cultivate more technological talents. The government plans to increase the proportion of technological investment to 2.5% of GDP in the next five years. In addition, the government also plans to increase technological investment in rural and impoverished areas to improve the level of technology in these areas.']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We also need to generate a system message."
      ],
      "metadata": {
        "id": "KC6iJzXjugJ-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def generate_system_message(prompt):\n",
        "\n",
        "    response = openai.ChatCompletion.create(\n",
        "        model=\"gpt-4\",\n",
        "        messages=[\n",
        "          {\n",
        "            \"role\": \"system\",\n",
        "            \"content\": \"You will be given a high-level description of the model we are training, and from that, you will generate a simple system prompt for that model to use. Remember, you are not generating the system message for data generation -- you are generating the system message to use for inference. A good format to follow is `Given $INPUT_DATA, you will $WHAT_THE_MODEL_SHOULD_DO.`.\\n\\nMake it as concise as possible. Include nothing but the system prompt in your response.\\n\\nFor example, never write: `\\\"$SYSTEM_PROMPT_HERE\\\"`.\\n\\nIt should be like: `$SYSTEM_PROMPT_HERE`.\"\n",
        "          },\n",
        "          {\n",
        "              \"role\": \"user\",\n",
        "              \"content\": prompt.strip(),\n",
        "          }\n",
        "        ],\n",
        "        temperature=temperature,\n",
        "        max_tokens=500,\n",
        "    )\n",
        "\n",
        "    return response.choices[0].message['content']\n",
        "\n",
        "system_message = generate_system_message(prompt)\n",
        "\n",
        "print(f'The system message is: `{system_message}`. Feel free to re-run this cell if you want a better result.')"
      ],
      "metadata": {
        "id": "xMcfhW6Guh2E",
        "outputId": "8cbf21d0-aaee-4496-89c8-35319dfccac3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The system message is: `Given a Chinese informative news article, translate and condense it into a well-understood English summary.`. Feel free to re-run this cell if you want a better result.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now let's put our examples into a dataframe and turn them into a final pair of datasets."
      ],
      "metadata": {
        "id": "G6BqZ-hjseBF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import json\n",
        "import pandas as pd\n",
        "\n",
        "# Initialize lists to store prompts and responses\n",
        "prompts = []\n",
        "responses = []\n",
        "\n",
        "# Parse out prompts and responses from examples\n",
        "for example in prev_examples:\n",
        "  try:\n",
        "    split_example = example.split('-----------')\n",
        "    prompts.append(split_example[1].strip())\n",
        "    responses.append(split_example[3].strip())\n",
        "  except:\n",
        "    pass\n",
        "\n",
        "# Create a DataFrame\n",
        "df = pd.DataFrame({\n",
        "    'prompt': prompts,\n",
        "    'response': responses\n",
        "})\n",
        "\n",
        "# Remove duplicates\n",
        "df = df.drop_duplicates()\n",
        "\n",
        "print('There are ' + str(len(df)) + ' successfully-generated examples.')\n",
        "\n",
        "# Initialize list to store training examples\n",
        "training_examples = []\n",
        "\n",
        "# Create training examples in the format required for GPT-3.5 fine-tuning\n",
        "for index, row in df.iterrows():\n",
        "    training_example = {\n",
        "        \"messages\": [\n",
        "            {\"role\": \"system\", \"content\": system_message.strip()},\n",
        "            {\"role\": \"user\", \"content\": row['prompt']},\n",
        "            {\"role\": \"assistant\", \"content\": row['response']}\n",
        "        ]\n",
        "    }\n",
        "    training_examples.append(training_example)\n",
        "\n",
        "# Save training examples to a .jsonl file\n",
        "with open('training_examples.jsonl', 'w') as f:\n",
        "    for example in training_examples:\n",
        "        f.write(json.dumps(example) + '\\n')"
      ],
      "metadata": {
        "id": "7CEdkYeRsdmB",
        "outputId": "0b8d7566-5d8a-4008-e247-08580c3ce7b3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "There are 49 successfully-generated examples.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Upload the file to OpenAI"
      ],
      "metadata": {
        "id": "KWTY6qVgXD_T"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "file_id = openai.File.create(\n",
        "  file=open(\"/content/training_examples.jsonl\", \"rb\"),\n",
        "  purpose='fine-tune'\n",
        ").id"
      ],
      "metadata": {
        "id": "4LjEUrI9XDgT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Train the model! You may need to wait a few minutes before running the next cell to allow for the file to process on OpenAI's servers."
      ],
      "metadata": {
        "id": "HmYRIq8dW9IR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "job = openai.FineTuningJob.create(training_file=file_id, model=\"gpt-3.5-turbo\")\n",
        "\n",
        "job_id = job.id"
      ],
      "metadata": {
        "id": "rdEyXmkoW80I"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Now, just wait until the fine-tuning run is done, and you'll have a ready-to-use model!\n",
        "\n",
        "Run this cell every 20 minutes or so -- eventually, you'll see a message \"New fine-tuned model created: ft:gpt-3.5-turbo-0613:xxxxxxxxxxxx\"\n",
        "\n",
        "Once you see that message, you can go to the OpenAI Playground (or keep going to the next cells and use the API) to try the model!"
      ],
      "metadata": {
        "id": "XUSX5QzmZMTd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "openai.FineTuningJob.list_events(id=job_id, limit=10)"
      ],
      "metadata": {
        "id": "45DJZ7hHaBx0",
        "outputId": "f882c2e2-cfa7-4767-82ef-7809de6a41a4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<OpenAIObject list at 0x7e83ade01c10> JSON: {\n",
              "  \"object\": \"list\",\n",
              "  \"data\": [\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-lUqPLRxTumnJaBtGLi1xDjpW\",\n",
              "      \"created_at\": 1699772043,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"The job has successfully completed\",\n",
              "      \"data\": {},\n",
              "      \"type\": \"message\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-d82L1DWEhzXeXP88eMgIFieS\",\n",
              "      \"created_at\": 1699772041,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"New fine-tuned model created: ft:gpt-3.5-turbo-0613:personal::8Jye0Nmn\",\n",
              "      \"data\": {},\n",
              "      \"type\": \"message\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-qm6hWjvlQqbyWebN1TzydUZq\",\n",
              "      \"created_at\": 1699772023,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 141/147: training loss=0.02\",\n",
              "      \"data\": {\n",
              "        \"step\": 141,\n",
              "        \"train_loss\": 0.02131996490061283,\n",
              "        \"train_mean_token_accuracy\": 0.9895833134651184\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-O3T2sD7KPJMV13qA3zUV7kLj\",\n",
              "      \"created_at\": 1699772002,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 131/147: training loss=0.00\",\n",
              "      \"data\": {\n",
              "        \"step\": 131,\n",
              "        \"train_loss\": 0.0026369730476289988,\n",
              "        \"train_mean_token_accuracy\": 1.0\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-wE2YurgNnQdUERCzKAF22NFc\",\n",
              "      \"created_at\": 1699771984,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 121/147: training loss=0.02\",\n",
              "      \"data\": {\n",
              "        \"step\": 121,\n",
              "        \"train_loss\": 0.018750470131635666,\n",
              "        \"train_mean_token_accuracy\": 1.0\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-CWhrl0GVakEcRSzE8VSOfo16\",\n",
              "      \"created_at\": 1699771965,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 111/147: training loss=0.05\",\n",
              "      \"data\": {\n",
              "        \"step\": 111,\n",
              "        \"train_loss\": 0.053563106805086136,\n",
              "        \"train_mean_token_accuracy\": 0.9896907210350037\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-r3KcRb3xd0DmSQZnsRgvZ1wG\",\n",
              "      \"created_at\": 1699771945,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 101/147: training loss=0.00\",\n",
              "      \"data\": {\n",
              "        \"step\": 101,\n",
              "        \"train_loss\": 6.516125722555444e-05,\n",
              "        \"train_mean_token_accuracy\": 1.0\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-lHy2x8wktAlzphqwvDoVknak\",\n",
              "      \"created_at\": 1699771926,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 91/147: training loss=0.15\",\n",
              "      \"data\": {\n",
              "        \"step\": 91,\n",
              "        \"train_loss\": 0.1539822816848755,\n",
              "        \"train_mean_token_accuracy\": 0.9642857313156128\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-9jvUB4ulnTx3ntWxfUO83dYl\",\n",
              "      \"created_at\": 1699771908,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 81/147: training loss=0.05\",\n",
              "      \"data\": {\n",
              "        \"step\": 81,\n",
              "        \"train_loss\": 0.049839481711387634,\n",
              "        \"train_mean_token_accuracy\": 0.9894737005233765\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    },\n",
              "    {\n",
              "      \"object\": \"fine_tuning.job.event\",\n",
              "      \"id\": \"ftevent-ofFmkhdlVBquCYMChU8DO9aP\",\n",
              "      \"created_at\": 1699771887,\n",
              "      \"level\": \"info\",\n",
              "      \"message\": \"Step 71/147: training loss=0.00\",\n",
              "      \"data\": {\n",
              "        \"step\": 71,\n",
              "        \"train_loss\": 0.00044341798638924956,\n",
              "        \"train_mean_token_accuracy\": 1.0\n",
              "      },\n",
              "      \"type\": \"metrics\"\n",
              "    }\n",
              "  ],\n",
              "  \"has_more\": true\n",
              "}"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Once your model is trained, run the next cell to grab the fine-tuned model name."
      ],
      "metadata": {
        "id": "91ihW2O27Phl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model_name_pre_object = openai.FineTuningJob.retrieve(job_id)\n",
        "model_name = model_name_pre_object.fine_tuned_model\n",
        "print(model_name)"
      ],
      "metadata": {
        "id": "eWBRBPh8aEzH",
        "outputId": "3f57f912-1745-4841-bcf4-e3a9d26964e4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "ft:gpt-3.5-turbo-0613:personal::8Jye0Nmn\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Let's try it out!"
      ],
      "metadata": {
        "id": "2OmZLoBX7oQM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "response = openai.ChatCompletion.create(\n",
        "    model=model_name,\n",
        "    messages=[\n",
        "      {\n",
        "        \"role\": \"system\",\n",
        "        \"content\": system_message,\n",
        "      },\n",
        "      {\n",
        "          \"role\": \"user\",\n",
        "          \"content\": df['prompt'].sample().values[0],\n",
        "      }\n",
        "    ],\n",
        ")\n",
        "\n",
        "response.choices[0].message['content']"
      ],
      "metadata": {
        "id": "uxbrmzc5dMuC",
        "outputId": "47a10d64-9b8f-4a61-827f-486e60f159fe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\"The Chinese government recently issued a new policy aimed at promoting the development of green buildings in the country. According to the policy, the government will provide a series of fiscal and tax incentives to encourage companies and individuals to invest in green buildings. The government stated that this policy will help improve China's energy efficiency, reduce carbon emissions, and also promote the development of China's green economy.\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}