This project provides a command-line interface (CLI) chat application using various NVIDIA models through the NVIDIA API. The application allows users to interact with different language models, each with specific parameters, and have conversations directly in the terminal.

• Supports multiple NVIDIA models with specific parameters.
• Interactive chat interface using rich for better terminal formatting.
• Configuration via environment variables.
• API key management for secure access to NVIDIA models.

• Python 3.6 or higher
• Dev account with Nvidia (it's free) for your API Key https://build.nvidia.com/explore/discover

1. Clone the Repository:

   git clone https://github.com/bigsk1/nvidia_cli_chat.git
   cd nvidia_cli_chat

2. Create a Virtual Environment:

   python3 -m venv venv
   source venv/bin/activate  # On Windows use `venv\Scripts\activate`

3. Install Dependencies:

   pip install -r requirements.txt

4. Set Up Environment Variables:

   Rename .env.sample to .env:

   API_KEY="your_single_api_key"
   
   MISTRAL_LARGE="mistralai/mistral-large"
   LLAMA3_70B="meta/llama3-70b-instruct"
   PHI_3_MINI_128K="microsoft/phi-3-mini-128k-instruct"
   ARCTIC="snowflake/arctic"
   GRANITE_34B_CODE="ibm/granite-34b-code-instruct"
   

   Replace your_single_api_key with your actual personal NVIDIA API key. You can add additional models by just copying the models format in .env and adding them to models.py

1. Run the Chat Interface:

   python main.py

2. Select a Model:

   You will be prompted to select a model by number. Each model has a specific name and description to help you choose the appropriate one for your needs.

3. Interact with the Model:

   • Type your messages in the terminal.
   • The model will respond with generated text based on your input.

4. Exit the Chat:

   • Type exit or quit to end the chat session.

nvidia_cli_chat/
├── main.py                # Main script to run the chat interface
├── api_handler.py         # Handles API requests to NVIDIA
├── chat_interface.py      # Manages the terminal chat interface
├── models.py              # Defines available models and their parameters
├── .env                   # Environment variables (not included in version control)
├── requirements.txt       # Project dependencies
└── README.md              # Project documentation

• main.py: The entry point of the application, which initializes the chat interface and manages the interaction loop.
• api_handler.py: Contains the NvidiaAPI class that handles requests to the NVIDIA API.
• chat_interface.py: Uses rich to create an interactive and formatted chat interface in the terminal.
• models.py: Defines the available models, their descriptions, and parameters. Allows users to select a model at runtime.
• .env: Stores environment variables including the API key and model identifiers.
• requirements.txt: Lists the Python packages required to run the application.

Code examples in terminal

This project is licensed under the MIT License.