add mkldnn softmax backward #17170

rongzha1 · 2019-12-25T08:19:19Z

Description

add mkldnn softmax backward implementation
unitest pass

Should fix #13365

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

@PatricZhao @TaoLv

TaoLv

Please take a look at #17152 and cache primitive for backward pass. Thanks!

src/operator/nn/mkldnn/mkldnn_ops-inl.h

src/operator/nn/mkldnn/mkldnn_softmax.cc

TaoLv · 2019-12-25T08:30:39Z

src/operator/nn/mkldnn/mkldnn_softmax.cc

+ { MKLDNN_ARG_DIFF_SRC, *out_mem.second },
+ };
+
+ stream->RegisterPrimArgs(bwd_pd, args);


will change it to cache primitive pd.

I mean here you need give a primitive not a primitive descriptor. Please check the definition of RegisterPrimArgs.

yes, you're right.
Although it is a pd, it call implicit constructor
inline primitive::primitive(const primitive_desc &pd) : primitive(pd.get()) {}
to get a primitive

I will make it more clearly by adding constructor before used.

TaoLv · 2019-12-25T08:31:17Z

src/operator/nn/softmax.cc

+ const std::vector<NDArray>& inputs,
+ const std::vector<OpReqType>& req,
+ const std::vector<NDArray>& outputs) {
+ // It seems MKLDNN softmax doesn't support training.


Can you elaborate?

Will remove this out-dated comments

TaoLv · 2019-12-25T08:32:03Z

src/operator/nn/softmax.cc

 inline static bool SoftmaxStorageType(const nnvm::NodeAttrs& attrs,
 const int dev_mask,
 DispatchMode* dispatch_mode,
 std::vector<int> *in_attrs,
 std::vector<int> *out_attrs) {
 const SoftmaxParam& param = nnvm::get<SoftmaxParam>(attrs.parsed);
- CHECK_EQ(in_attrs->size(), (param.use_length.value()) ? 2U : 1U);


Why remove this?

this check will result in backward check fail.
Will recover this check and add another function for backward.

pengzhao-intel · 2020-01-03T00:50:22Z

@rongzha1 please retrigger the PR @TaoLv could you help review again?

TaoLv · 2020-01-14T15:21:45Z

src/operator/nn/mkldnn/mkldnn_softmax.cc

+ std::shared_ptr<mkldnn::softmax_backward> bwd_;
+};
+
+typedef ParamOpSign<SoftmaxParam> MKLDNNSoftmaxSignature;


Same as L99?

exactly same, add extra one when rebase, will remove this line Thanks

TaoLv · 2020-01-14T15:23:39Z

src/operator/nn/mkldnn/mkldnn_softmax.cc

+void MKLDNNSoftmaxBackward(const nnvm::NodeAttrs& attrs,
+ const OpContext &ctx,
+ const std::vector<NDArray> &in_data,
+ const std::vector<OpReqType>& req,


Suggested change

const std::vector<OpReqType>& req,

const std::vector<OpReqType> &req,

TaoLv · 2020-01-14T15:37:09Z

src/operator/nn/mkldnn/mkldnn_softmax.cc

+ auto data_mem = in_data[1].GetMKLDNNData();
+ auto bwd = GetSoftmaxBwd(param, axis, in_data, out_data);
+
+ auto out_mem = CreateMKLDNNMem(out_data[0], bwd.pd.diff_src_desc(), req[0]);


Please check if you want to support req=kAddTo;

softmax backward primitive should support in-place calculation so no need to create additional buffer.

you are right, create mem is used to support kAddTo

Does the original softmax backward support kAddTo?

yes, original softmax bwd support kAddTo : SoftmaxGradCompute-->SoftmaxGrad -->KERNEL_ASSIGN(igrad[base + j*sa], Req, final_result);

pengzhao-intel · 2020-02-08T10:05:17Z

@TaoLv were all concerns resolved now?

* add mkldnn softmax backward * add primitive cache for softmax bwd * fix preci failed test * rm duplicate line

TaoLv suggested changes Dec 25, 2019

View reviewed changes

TaoLv added the MKLDNN label Dec 25, 2019

pengzhao-intel added this to In progress in CPU Performance and Quantization via automation Dec 26, 2019

TaoLv reviewed Jan 14, 2020

View reviewed changes

rongzha1 added 4 commits January 16, 2020 10:04

add mkldnn softmax backward

9c7ef9e

add primitive cache for softmax bwd

98efa1e

fix preci failed test

7a76d7e

rm duplicate line

05302b0

CPU Performance and Quantization automation moved this from In progress to Reviewer approved Feb 9, 2020

TaoLv approved these changes Feb 9, 2020

View reviewed changes

TaoLv merged commit 9883b99 into apache:master Feb 9, 2020

CPU Performance and Quantization automation moved this from Reviewer approved to Done Feb 9, 2020

TaoLv mentioned this pull request Feb 12, 2020

Fix storage type infer of softmax backward #17576

Merged

7 tasks

zheyuye pushed a commit to zheyuye/incubator-mxnet that referenced this pull request Feb 19, 2020

add mkldnn softmax backward (apache#17170)

d9f5165

* add mkldnn softmax backward * add primitive cache for softmax bwd * fix preci failed test * rm duplicate line

anirudh2290 pushed a commit to anirudh2290/mxnet that referenced this pull request May 29, 2020

add mkldnn softmax backward (apache#17170)

17b46c8

* add mkldnn softmax backward * add primitive cache for softmax bwd * fix preci failed test * rm duplicate line

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add mkldnn softmax backward #17170

add mkldnn softmax backward #17170

rongzha1 commented Dec 25, 2019 •

edited by TaoLv

Loading

TaoLv left a comment

TaoLv Dec 25, 2019

rongzha1 Dec 27, 2019

TaoLv Dec 27, 2019

rongzha1 Dec 27, 2019

rongzha1 Dec 27, 2019

TaoLv Dec 25, 2019

rongzha1 Dec 27, 2019

TaoLv Dec 25, 2019

rongzha1 Dec 27, 2019

pengzhao-intel commented Jan 3, 2020

TaoLv Jan 14, 2020

rongzha1 Jan 15, 2020

TaoLv Jan 14, 2020

rongzha1 Feb 8, 2020

TaoLv Jan 14, 2020

rongzha1 Jan 15, 2020

TaoLv Jan 20, 2020

rongzha1 Jan 21, 2020

pengzhao-intel commented Feb 8, 2020

	const std::vector<OpReqType>& req,
	const std::vector<OpReqType> &req,

add mkldnn softmax backward #17170

add mkldnn softmax backward #17170

Conversation

rongzha1 commented Dec 25, 2019 • edited by TaoLv Loading

Description

Checklist

Essentials

Changes

Comments

TaoLv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pengzhao-intel commented Jan 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pengzhao-intel commented Feb 8, 2020

rongzha1 commented Dec 25, 2019 •

edited by TaoLv

Loading