forgot some tts stuff

.gitignore

@@ -9,5 +9,5 @@ dist
 .DS_Store
 mpnet/
 baai/
-vits-piper-en_US-glados
 whisper
+espeak-ng-data

README.md

@@ -1,26 +1,41 @@
-# HELLO!
+<p align="center">
+  <img src="https://github.com/alexpinel/Dot/assets/93524949/9ab51fa9-3471-427f-b932-c001bac35346" alt="Dot App Banner">
+</p>
 
-![ezgif-4-b96c0b5548](https://github.com/alexpinel/Dot/assets/93524949/e5983c61-d59c-45ac-86f6-9d62cffaf37b)
+<p align="center">
+  <img src="https://img.shields.io/badge/license-GPL3.0-brightgreen.svg?style=flat-square" alt="License">
+  <img src="https://img.shields.io/github/v/release/alexpinel/Dot?style=flat-square" alt="GitHub release (latest by date)">
+  <img src="https://img.shields.io/github/commits-since/alexpinel/Dot/latest.svg?style=flat-square" alt="GitHub commits">
+  <img src="https://img.shields.io/github/stars/alexpinel/Dot.svg?style=social&label=Star&style=flat-square" alt="GitHub stars">
+</p>
 
-This is Dot, a standalone open source app meant for easy use of local LLMs and RAG in particular to interact with documents and files similarly to Nvidia's Chat with RTX. Dot itself is completely standalone and is packaged with all dependencies including a copy of Mistral 7B, this is to ensure the app is as accessible as possible and no prior knowledge of programming or local LLMs is required to use it. You can install the app (available for Apple Silicon and Windows) here: [Dot website ](https://dotapp.uk/)
+<p align="center">
+  <strong><a href="https://dotapp.uk/">Visit the Dot Website</a></strong>
+</p>
 
-### What does it do?
+---
 
-Dot can be used to load multiple documents into an llm and interact with them in a fully local environment through Retrieval Augmented Generation (RAG), supported documents are: pdf, docx, pptx, xlsx, and markdown. Apart from RAG, users can also switch to Big Dot for any interactions unrelated to their documents similarly to ChatGPT.
+## 🚀 About Dot
 
+Dot is a standalone, open-source application designed for seamless interaction with documents and files using local LLMs and Retrieval Augmented Generation (RAG). It is inspired by solutions like Nvidia's Chat with RTX, providing a user-friendly interface for those without a programming background. Pre-packaged with Mistral 7B, Dot ensures accessibility and simplicity right out of the box.
 
-https://github.com/alexpinel/Dot/assets/93524949/807fb58c-40e0-407e-afb3-a3813477ce9e
+https://github.com/alexpinel/Dot/assets/93524949/242ef635-b9f5-4263-8f9e-07bc040e3113
 
 
+### 📜 What does it do?
 
-### How does it work?
+Dot allows you to load multiple documents into an LLM and interact with them in a fully local environment. Supported document types include PDF, DOCX, PPTX, XLSX, and Markdown. Users can also engage with Big Dot for inquiries not directly related to their documents, similar to interacting with ChatGPT.
 
-Dot is built with Electron JS, but its main functionalities come from a bundled install of Python that contains all libraries and necessary files. A multitude of libraries are used to make everything work, but perhaps the most important to be aware of are: llama.cpp to run the LLM, FAISS to create local vector stores, and Langchain & Huggingface to setup the conversation chains and embedding process.
+### 🔧 How does it work?
 
-### Install
+Built with Electron JS, Dot encapsulates a comprehensive Python environment that includes all necessary libraries. The application leverages libraries such as FAISS for creating local vector stores, Langchain, llama.cpp & Huggingface for setting up conversation chains, and additional tools for document management and interaction.
 
-You can either install the packaged app in the [Dot website ](https://dotapp.uk/) or can set up the project for development, to do so follow these steps:
+## 📥 Install
 
+**To use Dot:**
+- Visit the [Dot website](https://dotapp.uk/) to download the application for Apple Silicon or Windows.
+
+**For developers:**
 - Clone the repository `$ https://github.com/alexpinel/Dot.git`
 - Install Node js and then run `npm install` inside the project repository, you can run `npm install --force` if you face any issues at this stage
 
@@ -44,18 +59,25 @@ Finally, download the Mistral 7B LLM from the following link and place it inside
 
 That's it! If you follow these steps you should be able to get it all running, please let me know if you are facing any issues :)
 
-### Future features I'd like to add:
+
+## 🌟 Future Features I'd Like to Add
 
 - Linux support
-- Ability to choose LLM
-- Image support would be cool
-- Increased awarnes of documents apart from merely their content
-- Loading individual files instead of selecting a folder  (This is really needed, some users get confused by this and I cannot blame them at all)
-- Increased security considerations, after all this is the whole point of using a local LLM
-- Support for more docs
-- Storing file databases, allowing users to quickly switch between groups of files without having to load them all again
-- idk, will find out along the way
-
-# Want to help?
-
-Please do! I am a busy student working on this as a side project so help is more than welcome!
+- Choice of LLM
+- Image file support
+- Enhanced document awareness beyond content
+- Simplified file loading (select individual files, not just folders)
+- Increased security measures for using local LLMs
+- Support for additional document types
+- Efficient file database management for quicker access to groups of files
+
+## 🤝 Want to Help?
+
+Contributions are highly encouraged! As a student managing this project on the side, any help is greatly appreciated. Whether it's coding, documentation, or feature suggestions, please feel free to get involved!
+
+---
+
+<div align="center">
+  <strong><a href="#top">Back to top</a></strong>
+</div>
+

llm/vits-piper-en_US-glados/README.md

@@ -0,0 +1,7 @@
+# Introduction
+
+This model is converted from
+https://github.com/dnhkng/GlaDOS/raw/main/models/glados.onnx
+
+See https://colab.research.google.com/drive/1m3Zr8H1RJaoZu4Y7hpQlav5vhtw3A513?usp=sharing
+for how the model is convered to sherpa-onnx

llm/vits-piper-en_US-glados/tokens.txt

@@ -0,0 +1,157 @@
+  3
+! 4
+" 150
+# 149
+$ 2
+' 5
+( 6
+) 7
+, 8
+- 9
+. 10
+0 130
+1 131
+2 132
+3 133
+4 134
+5 135
+6 136
+7 137
+8 138
+9 139
+: 11
+; 12
+? 13
+X 156
+^ 1
+_ 0
+a 14
+b 15
+c 16
+d 17
+e 18
+f 19
+g 154
+h 20
+i 21
+j 22
+k 23
+l 24
+m 25
+n 26
+o 27
+p 28
+q 29
+r 30
+s 31
+t 32
+u 33
+v 34
+w 35
+x 36
+y 37
+z 38
+æ 39
+ç 40
+ð 41
+ø 42
+ħ 43
+ŋ 44
+œ 45
+ǀ 46
+ǁ 47
+ǂ 48
+ǃ 49
+ɐ 50
+ɑ 51
+ɒ 52
+ɓ 53
+ɔ 54
+ɕ 55
+ɖ 56
+ɗ 57
+ɘ 58
+ə 59
+ɚ 60
+ɛ 61
+ɜ 62
+ɞ 63
+ɟ 64
+ɠ 65
+ɡ 66
+ɢ 67
+ɣ 68
+ɤ 69
+ɥ 70
+ɦ 71
+ɧ 72
+ɨ 73
+ɪ 74
+ɫ 75
+ɬ 76
+ɭ 77
+ɮ 78
+ɯ 79
+ɰ 80
+ɱ 81
+ɲ 82
+ɳ 83
+ɴ 84
+ɵ 85
+ɶ 86
+ɸ 87
+ɹ 88
+ɺ 89
+ɻ 90
+ɽ 91
+ɾ 92
+ʀ 93
+ʁ 94
+ʂ 95
+ʃ 96
+ʄ 97
+ʈ 98
+ʉ 99
+ʊ 100
+ʋ 101
+ʌ 102
+ʍ 103
+ʎ 104
+ʏ 105
+ʐ 106
+ʑ 107
+ʒ 108
+ʔ 109
+ʕ 110
+ʘ 111
+ʙ 112
+ʛ 113
+ʜ 114
+ʝ 115
+ʟ 116
+ʡ 117
+ʢ 118
+ʦ 155
+ʰ 145
+ʲ 119
+ˈ 120
+ˌ 121
+ː 122
+ˑ 123
+˞ 124
+ˤ 146
+̃ 141
+̧ 140
+̩ 144
+̪ 142
+̯ 143
+̺ 152
+̻ 153
+β 125
+ε 147
+θ 126
+χ 127
+ᵻ 128
+↑ 151
+↓ 148
+ⱱ 129

llm/vits-piper-en_US-glados/ttsProcessor.js

@@ -0,0 +1,90 @@
+const path = require('path');
+
+// Function to dynamically resolve the module paths from the local node_modules
+function requireLocalModule(moduleName) {
+    try {
+        // Attempt to require the module from the local node_modules directory
+        const modulePath = path.resolve(__dirname, 'node_modules', moduleName);
+        return require(modulePath);
+    } catch (error) {
+        console.error(`Error requiring module ${moduleName} from local node_modules directory:`, error);
+        throw error;
+    }
+}
+
+// Require the modules from the local node_modules directory
+const tar = requireLocalModule('tar');
+const sherpa_onnx = requireLocalModule('sherpa-onnx');
+
+function createOfflineTts() {
+    try {
+        console.log("Creating offline TTS...");
+
+        const currentDirectory = __dirname;
+
+        let offlineTtsVitsModelConfig = {
+            model: path.join(currentDirectory, 'en_US-glados.onnx'),
+            tokens: path.join(currentDirectory, 'tokens.txt'),
+            dataDir: path.join(currentDirectory, 'espeak-ng-data'),
+            lexicon: '',
+            dictDir: '',
+            noiseScale: 0.667,
+            noiseScaleW: 0.9,
+            lengthScale: 1.0,
+        };
+
+        let offlineTtsModelConfig = {
+            offlineTtsVitsModelConfig: offlineTtsVitsModelConfig,
+            numThreads: 1,
+            debug: 1,
+            provider: 'gpu',
+        };
+
+        let offlineTtsConfig = {
+            offlineTtsModelConfig: offlineTtsModelConfig,
+            ruleFsts: '',
+            ruleFars: '',
+            maxNumSentences: 2,
+        };
+
+        return sherpa_onnx.createOfflineTts(offlineTtsConfig);
+    } catch (error) {
+        console.error('Failed to create TTS:', error, error.stack);
+        process.exit(1);
+    }
+}
+
+process.on('message', async (data) => {
+    try {
+        console.log(`Received message to process: ${data.cmd}`);
+        const userDataPath = data.userDataPath;
+        const filePath = path.join(userDataPath, 'output.wav');
+
+        const tts = createOfflineTts();
+        console.log("TTS engine created, generating audio...");
+        const audio = tts.generate({
+            text: data.message,
+            sid: 0,
+            speed: 1.0
+        });
+        console.log(`Audio generated, saving to ${filePath}...`);
+        tts.save(filePath, audio);
+        tts.free();
+
+        console.log(`Posting back filePath: ${filePath}`);
+        process.send({ filePath: filePath });
+    } catch (error) {
+        console.error('Error processing TTS request:', error);
+        process.send({ error: error.message });
+    }
+});
+
+process.on('uncaughtException', error => {
+    console.error('Unhandled Exception:', error);
+    process.exit(1);
+});
+
+process.on('unhandledRejection', reason => {
+    console.error('Unhandled Rejection:', reason);
+    process.exit(1);
+});