Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

use MXNET_SAFE_ACCUMULATION for softmax accumulator #15037

Merged
merged 1 commit into from
May 22, 2019

Conversation

szha
Copy link
Member

@szha szha commented May 22, 2019

Description

use MXNET_SAFE_ACCUMULATION for softmax accumulator

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)

Changes

  • use MXNET_SAFE_ACCUMULATION for softmax accumulator

@abhinavs95
Copy link
Contributor

@szha why is the default value for MXNET_SAFE_ACCUMULATE=0?
When training a word language model (from #14722 ) I'm getting validation perplexity of 131 with the variable set to 1 compared to 140 when its set to zero.

@eric-haibin-lin
Copy link
Member

eric-haibin-lin commented Jul 26, 2019

@nswamy says it causes regression and backward compatibility should be kept

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants