1. Screenshots
2. Models
3. ChatGPT
4. Installation
5. Commands
6. Arguments
7. Keyboard

You can install it with pipx:

pipx install git+https://github.com/Merkoba/Meltdown --force

Which provides the meltdown command.

To install it with Vulkan support (GPU), you can do this:

CMAKE_ARGS="-DLLAMA_VULKAN=on" pipx install git+https://github.com/Merkoba/Meltdown --force

To install manually, use a virtual env and requirements.txt.

You can use scripts/venv.sh to automate this.

If you use AMD you might want to use this:

CMAKE_ARGS="-DLLAMA_VULKAN=on" scripts/venv.sh

There's a scripts/venv_amd.sh that does this.

To run the program, use run.sh in the root dir.

More information here.

You will need some models to play with.

Here's a good one you can use:

https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/blob/main/Meta-Llama-3-8B-Instruct.Q5_K_M.gguf

You can find more on that site.

ChatGPT is also supported.

You must first set the API key for it to work.

This can be done using the model menu: Set API Key.

Then pick the model you want: Use GPT Model.

Multi-modal models like llava 1.5 can be used.

Download the model gguf and the mmproj gguf (clip model):

https://huggingface.co/mys/ggml_llava-v1.5-7b/tree/main

Put those 2 files in the same directory.

Rename the clip model file to mmproj.gguf.

Set Mode to images.

Now you can use the File field to include a URL or path to an image.

And you can use the input to include text as normal.

Some widgets like File and Input accept drag and drop operations.

For instance you can drop a file to use its path.

Or text to use as input.

However, you need to have tkdnd installed in your system.

This is an extension for Tcl.

If you want to enable, it use the --drag-and-drop flag.

To enable the console use --show-console.

This allows you to send actions from the terminal that launched the program.

There are some keywords you can use in commands:

Name of the user.

Name of the AI.

Current date.

Current unix time in seconds.

Name of the current tab.

Random noun.

This is a special syntax to create uselinks.

These are used to prompt directly on click.