How to run Vanna from Local

[jagdishgg]

My server doesnt have internet connection and I want to run locally.

[zainhoda]

Do you have an LLM that’s locally runnable?

[jagdishgg]

Currently we dont have any LLM locally, might use codellama or Nsql, still searching option. Install vanna but will not be able to verify the vanna token as no internet on server

[LronDC]

I want to run Vanna locally too. After all, offline working environments are also relatively common. And I have run a few LLMs locally. Does Vanna have to do api-key certification?

[zainhoda]

You don't need to input a Vanna API key if you're using your own LLM and you're using vector storage that runs locally. See this notebook for an example:

https://vanna.ai/docs/snowflake-other-llm-chromadb.html

The example above uses ChromaDB, which runs mostly locally.

If you want to swap out for another vector database, that's also possible. That may be necessary if your server doesn't have an internet connection since ChromaDB downloads some weights to do embeddigs.