Performance Issue On Android

[mcmah309]

Hello,

First I'll say, really impressed by this library and looking forward to TTS!

I ran the example project on my android pixel 7 (Same one you used) and I am not seeing the same performance that was presented in the video here https://www.youtube.com/watch?v=SBaSpwXRz94 . I am getting about 1 word every 20 seconds.

I used the tinyllama-1.1b-chat-v1.0.Q2_K.gguf model found here.
https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/tree/main

I tried on a few more models and got the same issue.

Was there a specific model needed to achieved this? Or any specific configuration?

Run command: flutter run -d 28301FDH200MY4 --release

Device Info:

[
  {
    "name": "Pixel 7",
    "id": "28301FDH200MY4",
    "isSupported": true,
    "targetPlatform": "android-arm64",
    "emulator": false,
    "sdk": "Android 14 (API 34)",
    "capabilities": {
      "hotReload": true,
      "hotRestart": true,
      "screenshot": true,
      "fastStart": true,
      "flutterExit": true,
      "hardwareRendering": false,
      "startPaused": true
    }
  }
]

[BrutalCoding]

Hi there,

No, I did not use a specific model to achieve this. The one you suggested should work just as fast, but I remember that one of my commits (where I updated llama.cpp) slowed down the speed a lot to an unusable state. Later one, I did update binaries for llama.cpp again and that I believe fixed it.

Are you using this git branch directly (bad instructions, but more up-to-date) or the pub.dev package (easy to install, but outdated)?

Side note:
Not saying you're wrong, as I haven't verified it on my Pixel 7 now for a good month give or take. I have shifted focus from working on this project to fixing my financial situation since Dec 2023/Jan 2024. I shared my personal life update here last night: #16. Long story short: It's a bit hectic but I got a new job now, and I'd like to continue contributing to this project and keep things up-to-date. Just need to make sure that my open-source contributions and my contributions towards the commercial party is clearly communicated about and agreed upon beforehand.

[ArtemRedchych]

@mcmah309 Hello, I have the same problem as you, can you please provide information if you managed to solve this problem ? Thank you in advance