Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variability in Score Values Generated by Pretrained Model during Evaluation #183

Open
dimuthuanuraj opened this issue Mar 21, 2024 · 0 comments

Comments

@dimuthuanuraj
Copy link

Dear all,

I have extracted the scores for the test list while using a pretrained model and enabling the --eval option. I am a bit confused and unable to figure out why the same model gives me different score values for each run. This is because I am not training the model, only testing it.

Additionally, I am concerned that this may change the False Negative Rate (FNR) and False Positive Rate (FPR) values, leading to several different tuneThreshold values.

Could anyone provide an explanation for the difference in score values?

here is an example of different score values I obtained.

1st test run

-0.4855 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00008.wav
-0.5790 id10270/x6uYqmx31kE/00001.wav id10300/ize_eiCFEg0/00003.wav
-0.4624 id10270/x6uYqmx31kE/00001.wav id10270/GWXujl-xAVM/00017.wav
-0.5113 id10270/x6uYqmx31kE/00001.wav id10273/0OCW1HUxZyg/00001.wav
-0.3705 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00022.wav
-0.3966 id10270/x6uYqmx31kE/00001.wav id10284/Uzxv7Axh3Z8/00001.wav

2nd test run

-0.3934 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00008.wav
-0.4868 id10270/x6uYqmx31kE/00001.wav id10300/ize_eiCFEg0/00003.wav
-0.3250 id10270/x6uYqmx31kE/00001.wav id10270/GWXujl-xAVM/00017.wav
-0.4909 id10270/x6uYqmx31kE/00001.wav id10273/0OCW1HUxZyg/00001.wav
-0.3551 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00022.wav
-0.3532 id10270/x6uYqmx31kE/00001.wav id10284/Uzxv7Axh3Z8/00001.wav

3rd test run

-0.4118 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00008.wav
-0.4626 id10270/x6uYqmx31kE/00001.wav id10300/ize_eiCFEg0/00003.wav
-0.3340 id10270/x6uYqmx31kE/00001.wav id10270/GWXujl-xAVM/00017.wav
-0.4897 id10270/x6uYqmx31kE/00001.wav id10273/0OCW1HUxZyg/00001.wav
-0.3490 id10270/x6uYqmx31kE/00001.wav id10270/8jEAjG6SegY/00022.wav
-0.3424 id10270/x6uYqmx31kE/00001.wav id10284/Uzxv7Axh3Z8/00001.wav

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant