Repo of the code from the Medium article

This repo will work only with python version < 3.12

Create a new directory and a virtual environment

mkdir llmware
cd llmware
python -m venv venv

Activate the venv and install the following packages

source venv/bin/activate  #activate the venv on Mac/Linux
venv\Scripts\activate  #activate the venv on Windows
pip install llmware
pip install streamlit

Download then 2 files in the project directory:

• myapp.py
• logo.png

In the terminal, the the venv active run the following command

streamlit run myapp.py

During the first execution the application will download the models on your local machine
in the cache HuggingFace directory, something like this:

C:\Users\User\.cache\huggingface\hub   for the non quantized models
C:\Users\User\llmware_data\model_repo  for the GGUF files