-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recall, Precision, F1-Score are confusing #4
Comments
We have added an argument to support prediction labels along with scores. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I tried to check, if I used your algorithm correctly and if the data makes sense.
So I decided to compare your Recall, Precision and F1-Score to the values I got after using an outlier detection algorithm, but the values were very different. Then I realized that your metrics doesn't take the label defined but the algorithm as a parameter, only the ground-truth and the decision scores. After checking your algorithm I found, that you label data points as an anomaly if the decision score deviates from three times the standard deviation from the mean of the data. That was a bit confusing, since I thought I could use your metrics to evaluate how good my anomaly detection labeling was, but the anomaly detection algorithm I was using has a different strategy to label a data point as an outlier, so the values were different.
Is it possibly that you optionally also take the label from the algorithm as a parameter or at least explain that in the description, that the recall, precision, f1 might be different because of your own logic?
The strategy to use three times the standard deviation as a criterion for the labeling was also mentioned in your paper I remember, but as a user (especially if you didn't read the paper) it's still a bit confusing I found.
The text was updated successfully, but these errors were encountered: