Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[clojure]: add comp-metric based on CompositeEvalMetric #14553

Merged
merged 2 commits into from
Mar 29, 2019

Conversation

tolitius
Copy link
Contributor

Description

In order to be able to collect multiple metrics added a comp-metric function that can be used as:

=> (require '[org.apache.clojure-mxnet.eval-metric :as em])

=> (def metric (em/comp-metric [(em/accuracy)
                                (em/f1)
                                (em/top-k-accuracy 2)]))

=> (fit ... {:eval-metric metric})
Speed: 5.34 samples/sec	Train-accuracy=0.704545
Speed: 5.34 samples/sec	Train-f1=0.819692
Speed: 5.34 samples/sec	Train-top_k_accuracy=1.000000
Speed: 5.24 samples/sec	Train-accuracy=0.764881
Speed: 5.24 samples/sec	Train-f1=0.861308
Speed: 5.24 samples/sec	Train-top_k_accuracy=1.000000
...

comp metrics are read with the same (eval-metric/get):

=> (em/get metric)
{"accuracy" 0.71634614, "f1" 0.8284848, "top_k_accuracy" 1.0}

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http:https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • add comp-metric, tests, doc

Comments

  • to remain backwards compatible (eval-metric/get metric) would keep returning vector for anything but CompositeEvalMetric, for which it will return a map

@gigasquid
Copy link
Member

Looks very cool! Thanks! Look forward to reviewing it :)

@abhinavs95
Copy link
Contributor

@mxnet-label-bot add [Clojure]

Copy link
Member

@gigasquid gigasquid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks for the contribution. It's good to merge as soon as CI is green.

@gigasquid gigasquid added pr-awaiting-merge Review and CI is complete. Ready to Merge and removed pr-awaiting-merge Review and CI is complete. Ready to Merge labels Mar 29, 2019
@tolitius
Copy link
Contributor Author

tolitius commented Mar 29, 2019

thank you for lisping mxnet :)

@gigasquid gigasquid merged commit b20f08b into apache:master Mar 29, 2019
vdantu pushed a commit to vdantu/incubator-mxnet that referenced this pull request Mar 31, 2019
* [clojure]: add comp-metric based on CompositeEvalMetric

* [clojure]: add comp-metric test
ZhennanQin pushed a commit to ZhennanQin/incubator-mxnet that referenced this pull request Apr 3, 2019
* [clojure]: add comp-metric based on CompositeEvalMetric

* [clojure]: add comp-metric test
nswamy pushed a commit that referenced this pull request Apr 5, 2019
* [clojure]: add comp-metric based on CompositeEvalMetric

* [clojure]: add comp-metric test
haohuanw pushed a commit to haohuanw/incubator-mxnet that referenced this pull request Jun 23, 2019
* [clojure]: add comp-metric based on CompositeEvalMetric

* [clojure]: add comp-metric test
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants