-
Notifications
You must be signed in to change notification settings - Fork 701
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FeatureRequest: Streaming audio input #60
Comments
We don't have support for streaming audio right now. There's nothing stopping you from sending one request after another with all the audio chunks you've received so far, and each time ask how many complete chants have been seen. Or in a more "chat" style, after each new chunk ask how many new ones have been completed. |
We also need input audio streaming but for a less spiritual app. We're working on a WebRTC based terminal (https://github.com/tuzig/terminal7) and we want to let the user speak to the terminal and get assistance from Gemini. For example, when the user says: "Print all my GCP instances", Gemini's response - |
The API / product doesn't support this right now, so there's not much we can do in the cookbook unfortunately. You can file product feedback from within AI Studio, using the |
I have an audio clip where a person says a particular Matra once!
Like this - Om Namah Shivay - This is your input voice
Now, The person starts chanting the same mantra Over an over an without any stop
Om Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah ShivayOm Namah Shivay
Note that there is no fixed silence between each time it is being said.
I need to show the count of the number of times he has spoken it correctly in runtime, as he speaks.
How can i achieve this using python and gemini
Note that the mantra can be very different as well as very long
Currently I have developed a system that implements a websocket to read the chant continuously and the input audio is sent at the time of handshake. the stream is collected and and split of regular intervals (approximately equal to the length of input), dumped into a temporary wav file and sent to gemini along with the input audio. But there is a catch! The user can obviously modify his speed and it is not fixed that the audio will have a integer number of mantras chanted.
For example the audio chunks might be like
chunk 1: Om Namah Shivay Om Namah ShivayOm Namah
Chunk 2: Shivay Om Namah ShivayOm Namah Shivay
Here I want gemini to count the total chants as 5 (2 +0.5+0.5+2)
How can this be acheived using Gemini?
Below is the link to my repo!
https://github.com/Praj-17/Chant-Counter
I would really apprecialte any resources that could help me solve this problem in runtime!
The text was updated successfully, but these errors were encountered: