[v2.4.8] Add LM Studio and Ollama as local options (logancyang#230)

README.md

@@ -21,7 +21,7 @@ If you enjoy Copilot for Obsidian, please consider [sponsoring this project](htt
 - Chat with ChatGPT right inside Obsidian in the Copilot Chat window.
 - No repetitive login. Use your own API key (stored locally).
 - No monthly fee. Pay only for what you use.
-- Model selection of OpenAI, Azure and **local** models powered by LocalAI.
+- Model selection of OpenAI, Azure and **local** models powered by LM Studio and Ollama.
 - No need to buy ChatGPT Plus to use GPT-4.
 - No usage cap for GPT-4 like ChatGPT Plus.
 - One-click copying any message as markdown.
@@ -31,19 +31,15 @@ If you enjoy Copilot for Obsidian, please consider [sponsoring this project](htt
 - Easy commands to **simplify, emojify, summarize, translate, change tone, fix grammar, rewrite into a tweet/thread, count tokens** and more.
 - Set your own parameters like LLM temperature, max tokens, conversation context based on your need (**pls be mindful of the API cost**).
 - **User custom prompt**! You can *add, apply, edit, delete* your custom prompts, persisted in your local Obsidian environment! Be creative with your own prompt templates, sky is the limit!
-- **Local model** support for **offline chat and QA** using LocalAI. Talk to your notes without internet! (experimental feature)
+- **Local model** support for **offline chat** using LM Studio and Ollama (experimental feature).
 
-## 🎬 Video Demos
+## 🎬 Demos
 
-#### 🎉 NEW in v2.4.0: Local Copilot! No internet required!! 🎉
+#### 🎉 NEW in v2.4.8: Local Copilot Chat with LM Studio and Ollama! No internet required! 🎉🎉🎉
 
-<a href="https://www.youtube.com/watch?v=3yPVDI8wZcI" target="_blank"><img src="./images/thumbnail-local-copilot.png" width="700" /></a>
+<!-- <a href="https://www.youtube.com/watch?v=3yPVDI8wZcI" target="_blank"><img src="./images/thumbnail-local-copilot-lm-studio.png" width="700" /></a> -->
 
-**Please make sure you go through this [Step-by-step setup guide](./localai_setup.md) to setup Local Copilot on your device correctly!**
-
-I've got feedback about "Use Local Copilot" toggle being unnecesary, so it is removed in v2.4.1. Now, make sure your have `--cors` flag enabled in your LocalAI server (or .env `CORS=true` if you use Docker). Then simply fill in the OpenAI Proxy Base URL as "http:https://localhost:8080/v1" and restart the plugin to chat with your local models.
-
-When you are done, clear the OpenAI Proxy Base URL to switch back to non-local models.
+LM Studio and Ollama are the 2 best choices for running local models on your own machine. Please check out the super simple setup guide [here](local_copilot.md). Don't forget to flex your creativity in custom prompts using local models!
 
 #### 🤗 New to Copilot? Quick Guide for Beginners:
 
@@ -54,9 +50,7 @@ When you are done, clear the OpenAI Proxy Base URL to switch back to non-local m
 - Fix grammar and spelling, Summarize, Simplify, Emojify, Remove URLs
 - Generate glossary, table of contents
 - Translate to a language of your choosing
-- Change tone: professional, casual, straightforward, confident, friendly
-- Make longer/shorter
-- Rewrite into a tweet/thread
+- You can find all Copilot commands in your command palette
 
 #### 💬 User Custom Prompt: Create as Many Copilot Commands as You Like!
 
@@ -151,13 +145,15 @@ When opening an issue, please include relevant console logs. You can go to Copil
 
 ## 📝 Planned features (feedback welcome)
 - New modes
- - Chat mode (originally Conversation mode): You can now provide multiple notes at once for context in conversations, for LLMs with an extended context window.
- - QA mode: You can index any folder and perform question and answer sessions using a **fully local** index and Retrieval-Augmented Generation system.
-- Support embedded PDFs as context
-- Interact with a powerful AI agent that knows your vault. Explore, brainstorm and review ideas like never before!
+ - **Chat mode** (originally Conversation mode): You can now provide multiple notes at once as context in conversations, for LLMs with an extended context window.
+ - **QA mode**: You can **index any folder** and perform question and answer sessions using a **local** search index and Retrieval-Augmented Generation (RAG) system.
+- Support **embedded PDFs** as context
+- Interact with a **powerful AI agent** that knows your vault who can search, filter and use your notes as context to work with. Explore, brainstorm and research like never before!
 
 ## 🙏 Thank You
-If you are enjoying Copilot, please support my work here: https://www.buymeacoffee.com/logancyang
+Did you know that [even the timer on Alexa needs internet access](https://twitter.com/logancyang/status/1720929870635802738)? In this era of corporate-dominated internet, I still believe there's room for powerful tech that's focused on privacy. A great **local** AI agent in Obsidian is the ultimate form of this plugin. If you share my vision, please consider [sponsoring this project](https://github.com/sponsors/logancyang) or buying me coffees!
+
+<a href="https://www.buymeacoffee.com/logancyang" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 40px !important;width: 150px !important;" ></a>
 
 Please also help spread the word by sharing about the Copilot for Obsidian Plugin on Twitter, Reddit, or any other social media platform you use.
 

local_copilot.md

@@ -0,0 +1,43 @@
+# Local Copilot Setup Guide
+
+## LM Studio
+
+[LM Studio](https://lmstudio.ai/) has the best UI for running local models, it has support for Apple Silicon, Windows, and Linux (in beta). After you download the correct version of LM Studio to your machine, the first thing is to download a model. Find something small to start with, such as Mistral 7B, and work your way up if you have a beefy machine.
+
+A rule of thumb to determine how large a model you can run:
+- If you are on an Apple Silicon Mac, look at your RAM
+- If you are on a Windows PC with a GPU, look at your VRAM.
+
+After you set your model and preset, you are free to test it out in LM Studio's Chat tab, making sure everything is working. Now you can find the Local Server tab, make sure you have `CORS` enabled, turn on hardware acceleration based on the type of device you have, and click Start Server. This will enable Copilot for Obsidian to access it.
+
+Notice that LM Studio doesn't require the user to do anything in the terminal whatsoever. It is the most user-friendly way to run local models on the market now!
+
+Here's an example for Apple Metal macs. I can run 7B models blazingly fast on my tiny Macbook Air M1 2020 with 16GB RAM!
+
+<img src="./images/lm-studio.png" alt="LM Studio">
+
+Pick LM STUDIO (LOCAL) in the model dropdown, and start chatting!
+
+<img src="./images/lm-studio-model-pick.png" alt="LM Studio Model">
+
+## Ollama
+
+[Ollama](https://ollama.ai/) currently supports Mac and Linux, they mentioned that Windows is coming soon.
+
+Go to their website, download, install Ollama and its command line tool on your machine.
+
+You can download your model by running either `ollama run <model_name>` or `ollama pull <model_name>` in your terminal. The default model is Llama 2 7B. But here let me use Mistral 7B as an example again.
+
+The `ollama run mistral` command downloads and starts a chat with Mistral 7B right inside the terminal. But that is not what we want. We want a local server for our plugin.
+
+Now, **start the local server with `OLLAMA_ORIGINS=app:https://obsidian.md* ollama serve`, this will allow the Obsidian app to access the local server without CORS issues**.
+
+<img src="./images/ollama-serve.png" alt="Ollama">
+
+Inside Copilot settings, enter `mistral` under Ollama model
+
+<img src="./images/ollama-setting.png" alt="Ollama">
+
+Pick OLLAMA (LOCAL) in the model dropdown and start chatting!
+
+#### Now, go crazy with local models using your custom prompts!

manifest.json

@@ -1,7 +1,7 @@
 {
  "id": "copilot",
  "name": "Copilot",
- "version": "2.4.7",
+ "version": "2.4.8",
  "minAppVersion": "0.15.0",
  "description": "A ChatGPT Copilot in Obsidian.",
  "author": "Logan Yang",