🌏 English | 日本語
✨ OpenAI at Scale is a workshop by FastTrack for Azure in Microsoft team that helps customers to build and deploy simple ChatGPT UI application on Azure.
- Chat UI
- Configure system prompts and hyperparameters
- Authenticate with Azure Active Directory and get user information from Microsoft Graph
- Collect application logs with Azure Log Analytics
- Store prompt log data to Azure Cosmos DB
- OS - Windows 11, MacOS or Linux
⚠ For Windows client user, please use Ubuntu 20.04 LTS (Windows subsystem for Linux) to run this application.
⚠ GitHub Codespaces is supported as Linux envionment.
- Azure CLI (v4.28.1 or higher)
- Node.js (v16.20 or higher)
- Python (v3.9 or higher)
- git client
- Docker Desktop or any other Docker environment
- Docker is used for Visual Studio Code Dev Container.
- Azure subscription
- Resources
- Azure OpenAI Service
- Azure Active Directory application
- Azure Log Analytics
- (Optional) Azure Cosmos DB
- Resources
⚠ Free account is not supported
- Role
- Contributor role or higher for Azure subscription
- Permission to create Azure Active Directory application
- Permission to create Azure OpenAI Service
There are some ways to create Azure OpenAI Service, we recommend to use Azure Portal if you are not familiar with Azure CLI.
Before started you need to choose a location for your service. You can find the list of available locations here -> supported location is here : Supported locations
- Azure Portal : You can follow the official document to create Azure OpenAI Service.
- Azure CLI : You can use the following command to create Azure OpenAI Service.
command example
# Set environment variables
export SUBSCRIPTION_ID=<your subscription id>
export RESOURCE_GROUP=<your resource group name>
export LOCATION=eastus #hard coded to avoid confusion, you can change any avaiable location.
export AZURE_OPENAI_SERVICE=<your openai service name>
export AZURE_OPENAI_CHATGPT_DEPLOYMENT=<deployment name of your gpt-35-turbo model>
az login #check your subscription id
az account set --subscription $SUBSCRIPTION_ID
az group create --name $RESOURCE_GROUP --location $LOCATION
az cognitiveservices account create \
--name $AZURE_OPENAI_SERVICE \
--kind OpenAI \
--sku S0 \
--resource-group $RESOURCE_GROUP \
--location $LOCATION \
--yes
az cognitiveservices account deployment create \
-g $RESOURCE_GROUP \
-n $AZURE_OPENAI_SERVICE \
--deployment-name $AZURE_OPENAI_CHATGPT_DEPLOYMENT \
--model-name gpt-35-turbo \
--model-version "0301" \
--model-format OpenAI \
--scale-settings-scale-type "Standard"
Follow the steps in register your application to register your application.
- Select
Single-page application (SPA)
as platform type - The Redirect URI will be
https://localhost:5000
andhttps://localhost:5173
for local development - Keep the
Application (client) ID
andDirectory (tenant) ID
for later use
You can create a Azure Cosmos DB account by following instructions on the Azure docs here and please make sure you enable Analytical Store, more details can be found here.
- Select
Core (SQL)
as API - Container name will be
chat_log
and partition key will be/chat_session_id
You need to create .env
files from .env.sample
to set up your environment variables before you spin up your application.
app/frontend/.env
- This file will be used for authentication function by Azure Active Directory SDK.
# Azure Active Directory application
VITE_CLIENTID="<your client id>"
VITE_TENANTID="<your tenant id>"
app/backend/.env
- This file will be used for accessing to Azure OpenAI Service and Azure Cosmos DB.
# Azure OpenAI Service
AZURE_OPENAI_SERVICE="<your Azure OpenAI Service endpoint>"
OPENAI_API_KEY="<your Azure OpenAI Service key>"
AZURE_OPENAI_CHATGPT_DEPLOYMENT="<your model deployment>"
# (Optional) Azure Cosmos DB
AZURE_COSMOSDB_ENDPOINT="https://<account_name>.documents.azure.com:443/"
AZURE_COSMOSDB_KEY="<your Azure Cosmos DB access Key>"
AZURE_COSMOSDB_DB="< your Azure Cosmos DB database name>"
⚠ Please use Azure Key Vault to configure environment variables in production environments.
command examples to get environment variables from Azure CLI.
export RESOURCE_GROUP=<your resource group name>
export AZURE_OPENAI_SERVICE=<your openai service name>
export AZURE_OPENAI_CHATGPT_DEPLOYMENT=<deployment name of your gpt-35-turbo model>
export OPENAI_API_KEY=`az cognitiveservices account keys list \
-n $AZURE_OPENAI_SERVICE \
-g $RESOURCE_GROUP \
-o json \
| jq -r .key1`
Python is required to run the backend Flask application.
cd app/backend
python -m venv ./backend_env
source .backend_env/bin/activate #bash
pip install -r requirements.txt
cd app/backend
flask run --debug #hot reload
#python ./app.py
Node.js is required to run the frontend React application.
cd app/frontend
npm install
For development
npm run dev
For production
npm run build
It is used to optimize and reduce the size of all application files which are deployed in app/backend/static folder.
⚠ Before you run following command, you must run
npm run build
on app/frontend to set frontend files to backend static dir.
- Example of Azure App Service
-
Deploy app to Azure App Service with easist way.
cd app/backend az webapp up --runtime "python:3.10" --sku B1 -g <Resource Group Name>
-
Deploy Azure App Service Plan and Web App separately.
-
You can deploy an app with above command but the command doesn't allow to change detailed App Service Plan and Web App settings. So if you want to change these settings you can deploy it separately with following command.
-
Create Azure App Service Plan resources
az appservice plan create -g <Resource Group Name> --is-linux -n <App Service Plan Name> --sku <SKU Name> --location eastus
-
Create WebApp Resource on above App Service Plan
az webapp create -g <Resource Group Name> -n <WebApp Name> -p <App Service Plan Name> -r "python:3.10"
⚡️completely optional: if your system needs to add private endpoint and/or VNET integration, you can add it here with following options.
-
VNET Integration
# you need create vnet/subnet before execute this command az webapp create -g <Resource Group Name> -n <WebApp Name> -p <App Service Plan Name> -r "python:3.10" --vnet <VNET Name> --subnet <Subnet Name>
-
Private Endpoint
# you need create vnet/subnet webapp before execute this command az network private-endpoint create \ -n <PE Name> \ -g <Resource Group Name> \ --vnet-name <VNET Name> \ --subnet <Subnet Name> \ --connection-name <Private Endpoint Connection Name> \ --private-connection-resource-id /subscriptions/SubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Web/sites/<WebApp Name> \ --group-id sites
-
-
Update redirectURI at aadConfig.ts and rebuild frontend app
-
Update redirectURI with following FQDN, which is webapp endpoint.
az webapp config hostname list -g <Resource Group Name> --webapp-name <WebApp Name> -o json | jq '.[0].name'
-
Rebuild frontend
cd app/frontend npm run build
-
-
Before deployed webapp, you must change the environment variable with application settings of Azure App Service.
az webapp config appsettings set --name <Web App Name> -g <Resource Group Name> --settings SCM_DO_BUILD_DURING_DEPLOYMENT="true"
-
Deploy demo app to WebApp
cd app/backend zip -r deploy.zip . az webapp deploy -g <Resource Group Name> -n <Webapp Name> --src-path deploy.zip --type zip
-
After deployed webapp, you must change the environment variables with application settings of Azure App Service.
az webapp config appsettings set --name <Web App Name> -g <Resource Group Name> --settings OPENAI_API_KEY=<KEY> AZURE_OPENAI_CHATGPT_DEPLOYMENT=<Deployment Model Name> AZURE_OPENAI_SERVICE=<OpenAI Service Name>
-
-
-
Example of Log collection
- Deploy Azure Log Analytics workspace
export APP_SERIVCE=<your app service name> export LOCATION=<azure datacenter region - eastus, japaneast, etc...> export RESOURCE_GROUP=<your resource group name> export WORKSPACE=<your log analytics workspace name> export DIAGSETTINNG_NAME=<your diagnistics setting name (arbitary)> az monitor log-analytics workspace create --name $WORKSPACE --resource-group $RESOURCE_GROUP --location $LOCATION
- Enable diagnostics setting
export RESOURCE_ID=`az webapp show -g $RESOURCE_GROUP -n $APP_SERIVCE --query id --output tsv | tr -d '\r'` export WORKSPACE_ID=`az monitor log-analytics workspace show -g $RESOURCE_GROUP --workspace-name $WORKSPACE --query id --output tsv | tr -d '\r'` az monitor diagnostic-settings create \ --resource $RESOURCE_ID \ --workspace $WORKSPACE_ID \ -n $DIAGSETTINNG_NAME \ --logs '[{"category": "AppServiceAppLogs", "enabled": true},{"category": "AppServicePlatformLogs", "enabled": true},{"category": "AppServiceConsoleLogs", "enabled": true},{"category": "AppServiceAuditLogs", "enabled": true},{"category": "AppServiceHTTPLogs", "enabled": true}]'
The logging chat on Azure Cosmos DB section explains in detail on how chat messages can be logged into Azure Cosmos DB and used in deriving insights further downstream.
You can ask question and feedback about this repo on GitHub Issues.
- ChatGPT + Enterprise data with Azure OpenAI and Cognitive Search
- This repo is based on this sample code.
We are welcome your contribution from customers and internal Microsoft employees. Please see CONTRIBUTING. We appreciate all contributors to make this repo thrive!