Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multi-class confusion matrix metrics #532

Closed
wants to merge 2 commits into from

Conversation

feifjiang
Copy link
Contributor

@feifjiang feifjiang commented Dec 2, 2020

Added confusion matrix calculation for multi-class classification, including:

  1. confusion matrix for the top n most frequent labels
  2. misclassification per class

@salesforce-cla
Copy link

salesforce-cla bot commented Dec 2, 2020

Thanks for the contribution! Unfortunately we can't verify the commit author(s): FeiFei Jiang <f***@f***.i***.s***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Salesforce.com Contributor License Agreement and this Pull Request will be revalidated.

@feifjiang feifjiang changed the title Add confusion matrix metrics Add multi-class confusion matrix metrics Dec 2, 2020
@codecov
Copy link

codecov bot commented Dec 2, 2020

Codecov Report

Merging #532 (a99fa5a) into master (13ad9cd) will increase coverage by 86.77%.
The diff coverage is 95.23%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #532       +/-   ##
===========================================
+ Coverage        0   86.77%   +86.77%     
===========================================
  Files           0      347      +347     
  Lines           0    12018    +12018     
  Branches        0      399      +399     
===========================================
+ Hits            0    10429    +10429     
- Misses          0     1589     +1589     
Impacted Files Coverage Δ
...op/evaluators/OpMultiClassificationEvaluator.scala 95.59% <95.23%> (ø)
...rce/op/stages/impl/preparators/SanityChecker.scala 90.57% <0.00%> (ø)
...scala/com/salesforce/op/testkit/RandomVector.scala 90.47% <0.00%> (ø)
...op/stages/impl/feature/TimePeriodTransformer.scala 100.00% <0.00%> (ø)
...m/salesforce/op/aggregators/ExtendedMultiset.scala 75.00% <0.00%> (ø)
...in/scala/com/salesforce/op/testkit/RandomMap.scala 100.00% <0.00%> (ø)
...a/com/salesforce/op/cli/gen/ProjectGenerator.scala 87.50% <0.00%> (ø)
...n/scala/com/salesforce/op/testkit/RandomText.scala 100.00% <0.00%> (ø)
...a/com/salesforce/op/testkit/PartiallyDefined.scala 100.00% <0.00%> (ø)
...a/com/salesforce/op/test/FeatureTypeEquality.scala 80.00% <0.00%> (ø)
... and 338 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 13ad9cd...a99fa5a. Read the comment docs.

FeiFei Jiang and others added 2 commits December 3, 2020 16:51
replace tuple with case class for misclassification per category

fixed ModelSelectorSummaryTest

1) Use binary search impl from Scala 2)increased scoring time in OpWorkflowModelLocalTest 3)renaming confusion matrix calculation params

Fix writing of model in the local file system (salesforce#528)

test user cla:missing

update RecordInisghtsLOCOTest Threshold

Fix writing of model in the local file system (salesforce#528)

test user cla:missing

increased threshold to 1.0 to fix this flaky test

Fix writing of model in the local file system (salesforce#528)

(1)set the default value of topN classes 10 (2)added confMatrixClassIndices attribute
@tovbinm
Copy link
Collaborator

tovbinm commented Dec 4, 2020

@gerashegalov are you having fun?! ;)

@gerashegalov
Copy link
Contributor

@tovbinm yes. some fun to trick the CLA bot with @feifjiang

@gerashegalov gerashegalov mentioned this pull request Dec 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants