Use on own audio stream

[HeChengHui]

@SaoYear
Thank you for your work!
I want to be able to detect a unique audio cue in my video stream that captures audio. Assuming that i have trained a fine-tuned model, how can i go about using it in a causal manner?

[SaoYear]

Hi! Thanks for the interest and sorry for the late response!

The CRNN model for SED is causal while the ATST-SED is not. ATST-SED composes of a pretrained ATST model, which is based on the Transformer model, and since the self-attention modules in ATST is NOT a "masked" version. The ATST cannot be used in an online manner.

As for obtaining the audio cue, if you want to capture the audio cue in an offline way, you can refer to this issue.

Hope these help and please contact me if you have any other problem.

Nian

[HeChengHui]

@SaoYear

Thank you for the clarification.

The CRNN model for SED is causal

what model are you referring? is it a different repo?

[SaoYear]

Sorry,

the CRNN means the baseline model of the DCASE challenge 2021-2023, you can find it here.

This CRNN is also included in the ATST-SED architecture, which is why I mentioned it.