This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fix #15029
The rootcause of the problem is when NDArray is 1-d and the platform is GNU Linux, the backend implementation uses
__gnu_parallel:random_shuffle()
See: https://github.com/apache/incubator-mxnet/blob/master/src/operator/random/shuffle_op.cc#L53. This explains why crash did not happen in MacOS.The random_shuffle template defined in gcc (https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/parallel/random_shuffle.h#L384) is passing in
std::numeric_limits<uint32_t>::max()
to the __rng() function to generate random seed. Therefore, in our rng() function (https://github.com/apache/incubator-mxnet/blob/master/src/operator/random/shuffle_op.cc#L49) we cannot use our self defined data typeindex_t
and have to useuint32_t
to avoid integer overflow.Why this bug was not detected in our unit test? Because our unit test of shuffle only tests 2-D shape, which did not use parallel_shuffle in backend.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.