Welcome to the LLM RAG workshop! In this workshop, you will be deploying a pre-built solution that leverages a range of AWS services to implement an end to end Document Retrieval Q&A Chatbot
Login to Event Engine (https://dashboard.eventengine.run)
- Enter Your hashkey for login
- Enter your email and get one time password for login
- Select AWS console for access console environments
- Select Open Console
- You should now see the console
- Search for the IAM service in the search bar at the top of the console and create an IAM user
- Provide a user name, ex:LLM_Deployment
- Attached Admin policy to this user and create user
- Select this user again and click on the security credentials tab
- Find the access key section and create an access key
- Select the
Command Line Interface (CLI)
use case and check the confirmation box to create your access key - Copy the Access key and Secret access key to local text file (Be careful! the secret key is shown only once!)
- Create a Cloud9 environment
![image](https://private-user-images.githubusercontent.com/29404157/250327542-4790aa42-f26a-40c6-bb1c-2d165bcec8bb.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NDk0MDUsIm5iZiI6MTcyNDg0OTEwNSwicGF0aCI6Ii8yOTQwNDE1Ny8yNTAzMjc1NDItNDc5MGFhNDItZjI2YS00MGM2LWJiMWMtMmQxNjViY2VjOGJiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDEyNDUwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZkNGRhNjQ3ZjAzZmUxOTkzMmQ2Nzc0ZjkxMjZmZDVjOTNhMzg1NjNmMTk4NjFjYTI0YjM0ZmIxMGViYWJmMzEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.e63wGWjvzHejZTec1ruPui-w74lTmC6F03tUQly8ZjE)
-
Use
aws configure
and enter the Access key and Secret access key you just record before. leave the region by default -
Select force update
-
Download workshop package to cloud9 instance
wget https://jameschiang1001.s3.amazonaws.com/smart_search-v2-k.zip
-
Unzip the download file
unzip smart_search-v2-k.zip
-
Use cdk to deploy the AWS Lambda, Amazon Kendra and Amazon SageMaker service
cd smart_search-v2/deployment/ pip install -r requirements.txt
-
Export the related environment variables
export AWS_ACCOUNT_ID={your account} export AWS_REGION={your region}
Find our Account ID in the dropdown in the top right corner of the console:
Region:
-
Deploy the project using the Cloud Development Kit (CDK). This process will take around 30 mins
cdk bootstrap cdk deploy --all --require-approval never
- Download the archive containing a list of AWS white papers
cd ~/environment
wget https://jameschiang1001.s3.amazonaws.com/AWS_Whitepapers.zip
unzip AWS_Whitepapers.zip
-
Check the Amazon Kendra S3 data source, go to S3 console and look for a bucket name that looks like
smart-search-kendraxxxx
-
Go Back to your Cloud9 IDE, use the following command to upload the white papers to S3
aws s3 sync AWS_Whitepapers s3://{your-s3-bucket}/
- Look for the
SmartSearchNotebook
inSageMaker -> Notebook -> Notebook instances
, click Open Jupyter - Open LLM_Model/llm-english/code/inference.py and modify line 37 LLM_NAME value to TheBloke/vicuna-7B-1.1-HF (remember to save the file)
![image](https://private-user-images.githubusercontent.com/29404157/250341910-39d641d3-8b94-44b4-862f-184c03e0089c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NDk0MDUsIm5iZiI6MTcyNDg0OTEwNSwicGF0aCI6Ii8yOTQwNDE1Ny8yNTAzNDE5MTAtMzlkNjQxZDMtOGI5NC00NGI0LTg2MmYtMTg0YzAzZTAwODljLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDEyNDUwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU4MGFkMTFiNDYyMjlkNzZmM2RmYjcyN2UzYTczNWFhYWU3ZGRiYzFiOTg1MTBhMmQ5OTk3ZWY0YjUzODYxMGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.wAOav7XlgQEYhctvbEV5hOyc2rDUofrKtwGDw_znkMg)
-
Back to LLM_Model/llm-english/ and open llm_english_sagemaker_byos.ipynb. Run the code sequentially.The second step may take more than 15 mins.
-
Change chiness question to english to make sure the model can run well
![image](https://private-user-images.githubusercontent.com/29404157/250342631-3857c2d7-99c5-4dd9-8e47-8c8d86c3cd58.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NDk0MDUsIm5iZiI6MTcyNDg0OTEwNSwicGF0aCI6Ii8yOTQwNDE1Ny8yNTAzNDI2MzEtMzg1N2MyZDctOTljNS00ZGQ5LThlNDctOGM4ZDg2YzNjZDU4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDEyNDUwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU4NjkwNTYxODk2NGZkM2RjNDgxZTg0M2MwNTE2ODVlODU5ZTE0ZWMzN2ZmOWZiOTg1YzAxMDZhOGNiNGMzMTQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.SAtpJ-Yyp9w1ReYfxuyJwpSOTOUEmliO3TGe49qIAJk)
-
back to cloud9 console, select gradio-webpage.py under gradio-web. Change the url to your own url.
-
install gradio
pip install gradio
if you see following errors
please reinstall fastapi
pip install fastapi
-
running gradio for web service
python gradio-webpage.py
-
please check the public url. if you can see following page. the deployment is success.
Enter following questions for testing
- What is aws sagemaker
![image](https://private-user-images.githubusercontent.com/29404157/250347418-11f3c23b-e866-443a-8edf-df745acffd55.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NDk0MDUsIm5iZiI6MTcyNDg0OTEwNSwicGF0aCI6Ii8yOTQwNDE1Ny8yNTAzNDc0MTgtMTFmM2MyM2ItZTg2Ni00NDNhLThlZGYtZGY3NDVhY2ZmZDU1LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDEyNDUwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWIzZDUzY2U0MTViZTUyNzZmMzMzOWE3ODdkNDkyZDRkMWY5NzU0ZDRiZmYyNjY1MWRlYjZkYjViNWZjMTE0MGYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.x-_A3fFIf9chelrljiMGkwNXjO7zy8JJkEVuUBYQ6Qo)
![image](https://private-user-images.githubusercontent.com/29404157/250347552-f9d2aa5b-dcc8-4601-8fb7-d2f5d411598c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NDk0MDUsIm5iZiI6MTcyNDg0OTEwNSwicGF0aCI6Ii8yOTQwNDE1Ny8yNTAzNDc1NTItZjlkMmFhNWItZGNjOC00NjAxLThmYjctZDJmNWQ0MTE1OThjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDEyNDUwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWMyN2VjY2Y5M2Y0OTFhNjZmYjQ3OWYwMjQ5NGUyNjM4NWI3Yzk2ZGIyNzU1NDIyYWE0YTkzNDczYjYzNmMzODAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.0PgIGbVGOomKuHhfumRu3fVuzoBrt6bMH3ELpt1JZiw)
![image](https://private-user-images.githubusercontent.com/29404157/250347617-16916855-2d70-4179-9ae8-3ef27b38500d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ4NDk0MDUsIm5iZiI6MTcyNDg0OTEwNSwicGF0aCI6Ii8yOTQwNDE1Ny8yNTAzNDc2MTctMTY5MTY4NTUtMmQ3MC00MTc5LTlhZTgtM2VmMjdiMzg1MDBkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI4VDEyNDUwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTAwNzMwYjFjZGVmYTI4NjA3OWI3MWE0YWY4Y2M2MzhkZGM2Y2Y3Zjg4MjAyMjZiNWRhZWZkMGRmY2VjZDI5YWUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.07v8bDp-lMArBm4QLyfJOMRnQqhz_dO4yBdRaqnzJsk)
- Open a terminal in Jupyter
- Clone the workshop repository using the following terminal commands
cd SageMaker
git clone https://github.com/cliffordduke/llm-kendra-rag-workshop.git
- Navigate back to the Jupyter browser and into the cloned workshop folder, open the
kendra-langchain.ipynb
notebook. Make sure it us running theconda_python3
kernel
We will leverage the previous labs Kendra Index for demonstration purposes, in order for this notebook to work, please add AmazonKendraFullAccess
policy to the IAM role