This repo is a framework for a voice command Discord bot. Voice and text commands follow the command pattern, and adding new commands is simple.

This project is built and run using Docker. There are scripts in the repo to build the docker image and run a new container (buildandrundocker.ps1). Before building - go through Configuration to set up your .env file and Cloud Services for any cloud services you're using. Look at the top of the script for configuration parameters. The first time building will take a while.

The following parameters are available:

You need to create a .env file at the root directory of the bot and set the values marked as 'Required'. If .env values are set to use cloud services, those services must be set up correctly. See Cloud Services. The .env file holds the configuration values for the bot. The following table describes each value:

Rename the '.env.example' file to '.env' to get started.

Both TEXT_TO_SPEECH_METHOD and SPEECH_TO_TEXT_METHOD have a 'LOCAL' option. The container for Jarvis contains software for performing speech-to-text and text-to-speech, and selecting the 'LOCAL' option will use these local services.

These options are completely free and do not require API keys.

Depending on the specs of your system where the bot is running, both of these tasks could take a while to complete (on my system, using LOCAL speech-to-text and text-to-speech takes about 4 seconds for speech-to-text and 6 seconds for text-to-speech). Using cloud services is much quicker and more accurate than using the local services.

Another consideration is image size. Using the LOCAL services will pull models that significantly increase the size of the image. If you don't set any service to LOCAL, those models will not be pulled.

I was unable to get the local services installed correctly on a Raspberry Pi. If you're using the cloud services instead, then you should have no issues.

Google cloud services are used for speech-to-text (if GOOGLE is set as the TEXT_TO_SPEECH_METHOD) and text-to-speech (if GOOGLE is set as the TEXT_TO_SPEECH_METHOD). To use those services, you need to create a Google cloud account, create a new project and enable the text-to-speech and speech-to-text APIs. See APIs Dashboard for enabling Google APIs. See Creating an API key

Once the key is created, you need to download the google_key.json file into your keys/ folder in the root of the project - /keys/google_key.json

IBM Watson cloud services are used for speech-to-text (if IBM_WATSON is set as the speech-to-text method). See IBM Watson speech-to-text to create a speech-to-text service instance. After you created the instance, visit the service instance overview page. Click on the 'Manage' tab and find the 'Credentials' section. Click the 'Download' button to get the ibm_credentials.env credentials file.

Place the ibm_credentials.env file in the keys/ folder in the rood of the project - /keys/ibm_credentials.env

Spotify is used to find similar songs for the radio command. This is a free service, and just requires you to create an application in the Spotify developer portal. See Spotify Developer Portal to create an application. Once the application is created, copy the client ID and client secret to the .env file at the root of the project.

To run the bot, run the buildandrundocker.ps1 script in the root directory. This will build the docker image and run it in a new container. After the container is running, the bot should log in to the bot account, and listen for chat messages on any of the joined servers

After joining a voice channel, the bot will listen for a wake word from any user (each user audio stream is separate). The wake word is 'Jarvis' by default. If the wake word is heard on the user's audio stream, the bot will respond with "Yes {USER_NAME}". The bot will then listen for a command on that user audio stream. The command audio will be saved to ./recordings/{userId}.wav. After 2 seconds of silence, the command audio will be processed using speech-to-text. The first word of the command denotes the command type (e.g. 'play').

Some commands have special behavior is the wake word is detected:

• Play command - Stop the song if a song is currently playing
  • If a song is stopped, the bot will stop listening for commands (you have to say the wake word again for a command)
• Radio command - Stop the radio if the radio is currently playing
  • If the radio is stopped, the bot will stop listening for commands (you have to say the wake word again for a command)

Voice and text commands are handled through the same command handler. This means that any voice command can also be a text command. For example, you can say 'Jarvis. Play music best hits' or simply type ';;play music best hits' to achieve the same result.