Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Continuous mode not working #148

Closed
wants to merge 40 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
9fde685
Update README.md for Transcribe
vivekuppal Jun 29, 2023
1483fea
Merge pull request #1 from vivekuppal/vu-readme-updates
vivekuppal Jun 29, 2023
391d728
Allow usage without a valid OPEN API key. (#2)
vivekuppal Jun 29, 2023
ab4245d
Update README.md (#3)
vivekuppal Jun 29, 2023
ebb6f2f
Allow user to choose model. Add arguments to main file.
vivekuppal Jun 29, 2023
8f5a595
Code clean up, add linting. (#4)
vivekuppal Jun 30, 2023
59d5c91
UI Text Chronology (#5)
vivekuppal Jun 30, 2023
f772bb8
Update readme with Enhancements. Allow copy of text from UI window. R…
vivekuppal Jun 30, 2023
87a38b1
Save conversation to text. (#9)
vivekuppal Jun 30, 2023
65d6dcf
Add Contextual Information to Responses (#11)
vivekuppal Jun 30, 2023
d1b3c45
Allow users to pause audio transcription. Change the default for gett…
vivekuppal Jul 3, 2023
cfca51a
Update main.py (#15)
abhinavuppal1 Jul 11, 2023
152bad3
Code reorg to separate UI code (#16)
vivekuppal Jul 12, 2023
addf17f
Add support for multiple languages (#18)
vivekuppal Jul 12, 2023
e5cda88
Easy install for non developers on windows (#20)
vivekuppal Jul 18, 2023
9896c1c
Disabled winrar UI (#22)
Adarsha-gg Jul 18, 2023
901501b
When using API, we do not need to specify language, absorb the lang p…
vivekuppal Jul 18, 2023
bd48b61
Language combo fix (#26)
Adarsha-gg Jul 19, 2023
7c9ca88
Added gdrive (#27)
Adarsha-gg Jul 19, 2023
2429c97
Allow usage of API Key in installed version of Transcribe (#28)
vivekuppal Jul 19, 2023
12ef846
updated the drive link (#30)
Adarsha-gg Jul 20, 2023
4be26c7
Add a duration class to easily measure the time taken for an operatio…
vivekuppal Jul 21, 2023
6e53b31
--api option was not working correctly (#34)
vivekuppal Jul 21, 2023
bd42b8c
Initial unit tests for the speech recognition library (#36)
vivekuppal Jul 24, 2023
af87eff
user reported defect fixes. (#39)
vivekuppal Jul 26, 2023
26cfaad
Optimize LLM usage (#40)
vivekuppal Jul 26, 2023
f8d5857
Bug fixes for exceptions observed during usage. Add further plumbing …
vivekuppal Jul 27, 2023
1356a78
Add logging infrastructure (#42)
vivekuppal Jul 27, 2023
a1cc48b
Get Response from LLM on demand (#44)
vivekuppal Jul 28, 2023
ea5f392
Models from open ai site (#43)
Adarsha-gg Jul 28, 2023
b4e03a4
List all active devices (#45)
vivekuppal Aug 1, 2023
85d09ed
Allow user to select input, output audio devices (#48)
vivekuppal Aug 21, 2023
28d1e9a
Disable mic speaker selectively (#49)
vivekuppal Aug 23, 2023
e48bdb8
Add Audio Response for LLM generated content (#50)
vivekuppal Aug 27, 2023
6baa77f
Update, upload latest binaries (#54)
Adarsha-gg Aug 30, 2023
fa55416
Multiturn prompts, bug fixes (#55)
vivekuppal Sep 5, 2023
ce5a1e1
Allow enable/disable speaker and microphone from UI (#56)
Adarsha-gg Sep 6, 2023
e445856
Update gdrive link (#58)
Adarsha-gg Sep 7, 2023
b50f58c
Bring readme up to date with current functionality. Describe content …
vivekuppal Sep 8, 2023
a7ea2cc
Continuous mode broke after updates to the UI.
vivekuppal Sep 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Code reorg to separate UI code (#16)
code reorg to separate some of the UI code
  • Loading branch information
vivekuppal committed Jul 12, 2023
commit 152bad382b407c8a7fd25a9d1df2d720e1731b1b
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
## ➕ Enhancements from base repository ➕
- Do not need Open AI key, paid Open AI account to use the complete functionality
- Allow users to add contextual information to provide customized responses to conversation
- Allows usage of different models for transcription using command line arguments
- Allow to pause audio transcription
- Transcribe audio of any video
- Preserve all conversation text in UI
Expand Down
13 changes: 13 additions & 0 deletions interactions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import subprocess
import socket


def create_params() -> dict:
git_version = subprocess.check_output(
['git', 'rev-parse', '--short', 'HEAD']).decode("utf-8").strip()
hostname = socket.gethostname()
arg_dict = {
'version': git_version,
'hostname': hostname
}
return arg_dict
122 changes: 27 additions & 95 deletions main.py
Original file line number Diff line number Diff line change
@@ -1,104 +1,23 @@
import threading
import argparse
from argparse import RawTextHelpFormatter
import queue
import time
import requests
from AudioTranscriber import AudioTranscriber
from GPTResponder import GPTResponder
import prompts
import customtkinter as ctk
import AudioRecorder
import queue
import time
import torch
import TranscriberModels
import subprocess
import pyperclip


def write_in_textbox(textbox, text):
textbox.delete("0.0", "end")
textbox.insert("0.0", text)


def update_transcript_UI(transcriber, textbox):
transcript_string = transcriber.get_transcript()
write_in_textbox(textbox, transcript_string)
textbox.see("end")
textbox.after(300, update_transcript_UI, transcriber, textbox)


def update_response_UI(responder, textbox, update_interval_slider_label,
update_interval_slider, freeze_state):
if not freeze_state[0]:
response = responder.response

textbox.configure(state="normal")
write_in_textbox(textbox, response)
textbox.configure(state="disabled")

update_interval = int(update_interval_slider.get())
responder.update_response_interval(update_interval)
update_interval_slider_label.configure(text=f"Update interval: \
{update_interval} seconds")

textbox.after(300, update_response_UI, responder, textbox,
update_interval_slider_label, update_interval_slider,
freeze_state)


def clear_context(transcriber, audio_queue):
transcriber.clear_transcript_data()
with audio_queue.mutex:
audio_queue.queue.clear()


def create_ui_components(root):
ctk.set_appearance_mode("dark")
ctk.set_default_color_theme("dark-blue")
root.title("Transcribe")
root.configure(bg='#252422')
root.geometry("1000x600")

font_size = 20

transcript_textbox = ctk.CTkTextbox(root, width=300, font=("Arial", font_size),
text_color='#FFFCF2', wrap="word")
transcript_textbox.grid(row=0, column=0, padx=10, pady=20, sticky="nsew")

response_textbox = ctk.CTkTextbox(root, width=300, font=("Arial", font_size),
text_color='#639cdc', wrap="word")
response_textbox.grid(row=0, column=1, padx=10, pady=20, sticky="nsew")
response_textbox.insert("0.0", prompts.INITIAL_RESPONSE)

freeze_button = ctk.CTkButton(root, text="Suggest Response", command=None)
freeze_button.grid(row=1, column=1, padx=10, pady=3, sticky="nsew")

update_interval_slider_label = ctk.CTkLabel(root, text="", font=("Arial", 12),
text_color="#FFFCF2")
update_interval_slider_label.grid(row=2, column=1, padx=10, pady=3, sticky="nsew")

update_interval_slider = ctk.CTkSlider(root, from_=1, to=10, width=300, height=20,
number_of_steps=9)
update_interval_slider.set(2)
update_interval_slider.grid(row=3, column=1, padx=10, pady=10, sticky="nsew")

copy_button = ctk.CTkButton(root, text="Copy Audio Transcript", command=None)
copy_button.grid(row=2, column=0, padx=10, pady=3, sticky="nsew")

save_file_button = ctk.CTkButton(root, text="Save Audio Transcript to File", command=None)
save_file_button.grid(row=3, column=0, padx=10, pady=3, sticky="nsew")

transcript_button = ctk.CTkButton(root, text="Pause Transcript", command=None)
transcript_button.grid(row=4, column=0, padx=10, pady=3, sticky="nsew")

# Order of returned components is important. For adding new components add new components
# to the end
return [transcript_textbox, response_textbox, update_interval_slider,
update_interval_slider_label, freeze_button, copy_button,
save_file_button, transcript_button]
import interactions
import ui
from requests.exceptions import ConnectionError


def main():

# Set up all arguments
cmd_args = argparse.ArgumentParser(description='Command Line Arguments for Transcribe',
formatter_class=RawTextHelpFormatter)
Expand All @@ -109,10 +28,15 @@ def main():
default='tiny',
help='Specify the model to use for transcription.'
'\nBy default tiny model is part of the install.'
'\nbase model has to be downloaded from the link https://drive.google.com/file/d/1E44DVjpfZX8tSrSagaDJXU91caZOkwa6/view?usp=drive_link'
'\nsmall model has to be downloaded from the link https://drive.google.com/file/d/1vhtoZCwfYGi5C4jK1r-QVr5GobSBnKiH/view?usp=drive_link'
'\nbase model has to be downloaded from the link \
https://drive.google.com/file/d/1E44DVjpfZX8tSrSagaDJXU91caZOkwa6/view?usp=drive_link'
'\nsmall model has to be downloaded from the link \
https://drive.google.com/file/d/1vhtoZCwfYGi5C4jK1r-QVr5GobSBnKiH/view?usp=drive_link'
'\nOpenAI has more models besides the ones specified above.'
'\nThose models are prohibitive to use on local machines because of memory requirements.')
'\nThose models are prohibitive to use on local machines because \
of memory requirements.')
cmd_args.add_argument('-e', '--experimental', action='store_true', help='Experimental command\
line argument. Behavior is undefined.')
args = cmd_args.parse_args()

try:
Expand All @@ -124,8 +48,16 @@ def main():
ffmpeg and try again.")
return

query_params = interactions.create_params()
try:
response = requests.get("http:https://127.0.0.1:5000/ping", params=query_params)
if response.status_code != 200:
print(f'Error received: {response}')
except ConnectionError:
print('Operating as a standalone client')

root = ctk.CTk()
ui_components = create_ui_components(root)
ui_components = ui.create_ui_components(root)
transcript_textbox = ui_components[0]
response_textbox = ui_components[1]
update_interval_slider = ui_components[2]
Expand Down Expand Up @@ -171,7 +103,7 @@ def main():

# Add the clear transcript button to the UI
clear_transcript_button = ctk.CTkButton(root, text="Clear Audio Transcript",
command=lambda: clear_context(transcriber, audio_queue))
command=lambda: ui.clear_context(transcriber, audio_queue))
clear_transcript_button.grid(row=1, column=0, padx=10, pady=3, sticky="nsew")

freeze_state = [True] # Using list to be able to change its content inside inner functions
Expand Down Expand Up @@ -204,9 +136,9 @@ def set_transcript_state():
{update_interval_slider.get()} \
seconds")

update_transcript_UI(transcriber, transcript_textbox)
update_response_UI(responder, response_textbox, update_interval_slider_label,
update_interval_slider, freeze_state)
ui.update_transcript_ui(transcriber, transcript_textbox)
ui.update_response_ui(responder, response_textbox, update_interval_slider_label,
update_interval_slider, freeze_state)

root.mainloop()

Expand Down
101 changes: 101 additions & 0 deletions ui.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
import customtkinter as ctk
import AudioTranscriber
import prompts


def write_in_textbox(textbox: ctk.CTkTextbox, text: str):
"""Update the text of textbox with the given text
Args:
textbox: textbox to be updated
text: updated text
"""
textbox.delete("0.0", "end")
textbox.insert("0.0", text)


def update_transcript_ui(transcriber: AudioTranscriber, textbox: ctk.CTkTextbox):
"""Update the text of transcription textbox with the given text
Args:
transcriber: AudioTranscriber Object
textbox: textbox to be updated
"""
transcript_string = transcriber.get_transcript()
write_in_textbox(textbox, transcript_string)
textbox.see("end")
textbox.after(300, update_transcript_ui, transcriber, textbox)


def update_response_ui(responder, textbox, update_interval_slider_label,
update_interval_slider, freeze_state):
"""Update the text of response textbox with the given text
Args:
textbox: textbox to be updated
text: updated text
"""
if not freeze_state[0]:
response = responder.response

textbox.configure(state="normal")
write_in_textbox(textbox, response)
textbox.configure(state="disabled")

update_interval = int(update_interval_slider.get())
responder.update_response_interval(update_interval)
update_interval_slider_label.configure(text=f"Update interval: \
{update_interval} seconds")

textbox.after(300, update_response_ui, responder, textbox,
update_interval_slider_label, update_interval_slider,
freeze_state)


def clear_context(transcriber, audio_queue):
transcriber.clear_transcript_data()
with audio_queue.mutex:
audio_queue.queue.clear()


def create_ui_components(root):
ctk.set_appearance_mode("dark")
ctk.set_default_color_theme("dark-blue")
root.title("Transcribe")
root.configure(bg='#252422')
root.geometry("1000x600")

font_size = 20

transcript_textbox = ctk.CTkTextbox(root, width=300, font=("Arial", font_size),
text_color='#FFFCF2', wrap="word")
transcript_textbox.grid(row=0, column=0, padx=10, pady=20, sticky="nsew")

response_textbox = ctk.CTkTextbox(root, width=300, font=("Arial", font_size),
text_color='#639cdc', wrap="word")
response_textbox.grid(row=0, column=1, padx=10, pady=20, sticky="nsew")
response_textbox.insert("0.0", prompts.INITIAL_RESPONSE)

freeze_button = ctk.CTkButton(root, text="Suggest Response", command=None)
freeze_button.grid(row=1, column=1, padx=10, pady=3, sticky="nsew")

update_interval_slider_label = ctk.CTkLabel(root, text="", font=("Arial", 12),
text_color="#FFFCF2")
update_interval_slider_label.grid(row=2, column=1, padx=10, pady=3, sticky="nsew")

update_interval_slider = ctk.CTkSlider(root, from_=1, to=10, width=300, height=20,
number_of_steps=9)
update_interval_slider.set(2)
update_interval_slider.grid(row=3, column=1, padx=10, pady=10, sticky="nsew")

copy_button = ctk.CTkButton(root, text="Copy Audio Transcript", command=None)
copy_button.grid(row=2, column=0, padx=10, pady=3, sticky="nsew")

save_file_button = ctk.CTkButton(root, text="Save Audio Transcript to File", command=None)
save_file_button.grid(row=3, column=0, padx=10, pady=3, sticky="nsew")

transcript_button = ctk.CTkButton(root, text="Pause Transcript", command=None)
transcript_button.grid(row=4, column=0, padx=10, pady=3, sticky="nsew")

# Order of returned components is important.
# Add new components to the end
return [transcript_textbox, response_textbox, update_interval_slider,
update_interval_slider_label, freeze_button, copy_button,
save_file_button, transcript_button]