MusicGen Remixer is an app based on MusicGen Chord. Users can upload a music track with vocals, type in the text description prompt, and the app will create a new background track based on the input and then make a remixed music output. This Jupyter notebook breaks down the process of MusicGen Remixer, by calling separate Replicate API calls and processing the outputs of the API calls.

Works on Python>=3.8 && <3.11.

pip install -r requirements.txt

To call the APIs from Replicate, you must login to Replicate after installing replicate python package with pip. You can find your Replicate API tokens here.

export REPLICATE_API_TOKEN=<your token>

python musicgen_remixer.py --prompt="bossa nova" --audio_path=/path/to/your/audio/input.mp3

• --prompt : The prompt to use for generating the remix.
• --audio_path : The path to the audio file to remix.
• --model_version : The version of the model to use for generating the remix.
  • Default : chord
  • Options : [chord, chord-large, stereo-chord, stereo-chord-large]
• --beat_sync_threshold : The threshold for beat synchronization. If None, beat synchronization is automatically set to 1.1/(bpm/60).
  • Default : None
• --upscale : Whether to upscale the audio to 48 kHz. (boolean Flag)
  • Default : False
• --mix_weight : The weight for the generated instrumental track when mixing with the vocal.(0~1)
  • Default : 0.7
• --output_path : The path to save the output audio file.
  • Default : output

• Chord recognition from audio file is performed using BTC model, by Jonggwon Park.
  • Paper : A Bi-Directional Transformer for Musical Chord Recognition
• Vocal dropping is implemented using Meta's demucs.
• Downbeat tracking and BPM retrieval is perfromed using All-In-One Music Structure Analyzer by Taejun Kim.
  • Paper : All-In-One Metrical And Functional Structure Analysis With Neighborhood Attentions on Demixed Audio
• Beat-syncing is performed with PyTSMod by MAC Lab @KAIST

• All code in this repository is licensed under the Apache License 2.0 license.