Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Fix #13521 #13537

Merged
merged 2 commits into from
Dec 5, 2018
Merged

Fix #13521 #13537

merged 2 commits into from
Dec 5, 2018

Conversation

zhreshold
Copy link
Member

@zhreshold zhreshold commented Dec 4, 2018 •
edited
Loading

Description

Fix #13521
Python has bug that causing multiprocessing.pool not termniated correctly after gabage collection, this may result in too many dangling processes.

Fix the problem by manually terminate the pools.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http:https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@zhreshold zhreshold requested a review from szha as a code owner December 4, 2018 21:33
@zhreshold
Copy link
Member Author

@eric-haibin-lin Could you help review and merge? Thx

@zhreshold zhreshold merged commit f6b4665 into apache:master Dec 5, 2018
@zhreshold zhreshold deleted the release-mp-pool branch December 5, 2018 21:39
zhreshold added a commit that referenced this pull request Dec 5, 2018
@zhreshold
Fix #13521 (#13537)
d63fdfd 
* fix pool release

* fix
TaoLv added a commit that referenced this pull request Dec 6, 2018
@TaoLv
Revert "Fix #13521 (#13537)"
e527167 
This reverts commit f6b4665.
TaoLv added a commit that referenced this pull request Dec 6, 2018
@TaoLv
Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated L…
29885c5 
…icense file" (#13558)

* Revert "Chi_square_check for discrete distribution fix (#13543)"

This reverts commit cf6e8cb.

* Revert "Updated docs for randint operator (#13541)"

This reverts commit e0ff3c3.

* Revert "Simplifications and some fun stuff for the MNIST Gluon tutorial (#13094)"

This reverts commit 8bbac82.

* Revert "Fix #13521 (#13537)"

This reverts commit f6b4665.

* Revert "Add a retry to qemu_provision (#13551)"

This reverts commit f6f8401.

* Revert "[MXNET-769] Use MXNET_HOME in a tempdir in windows to prevent access denied due t… (#13531)"

This reverts commit bd8e0f8.

* Revert "[MXNET-1249] Fix Object Detector Performance with GPU (#13522)"

This reverts commit 1c8972c.

* Revert "Fixing a 404 in the ubuntu setup doc (#13542)"

This reverts commit cb0db29.

* Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file (#13478)"

This reverts commit 40db619.
zhaoyao73 pushed a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018
@zhreshold @zhaoyao73
Fix apache#13521 (apache#13537)
9ceae0d 
* fix pool release

* fix
zhaoyao73 pushed a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018
@TaoLv @zhaoyao73
Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated L…
ad7ba29 
…icense file" (apache#13558)

* Revert "Chi_square_check for discrete distribution fix (apache#13543)"

This reverts commit cf6e8cb.

* Revert "Updated docs for randint operator (apache#13541)"

This reverts commit e0ff3c3.

* Revert "Simplifications and some fun stuff for the MNIST Gluon tutorial (apache#13094)"

This reverts commit 8bbac82.

* Revert "Fix apache#13521 (apache#13537)"

This reverts commit f6b4665.

* Revert "Add a retry to qemu_provision (apache#13551)"

This reverts commit f6f8401.

* Revert "[MXNET-769] Use MXNET_HOME in a tempdir in windows to prevent access denied due t… (apache#13531)"

This reverts commit bd8e0f8.

* Revert "[MXNET-1249] Fix Object Detector Performance with GPU (apache#13522)"

This reverts commit 1c8972c.

* Revert "Fixing a 404 in the ubuntu setup doc (apache#13542)"

This reverts commit cb0db29.

* Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file (apache#13478)"

This reverts commit 40db619.
zhaoyao73 added a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018
@zhaoyao73
Merge remote-tracking branch 'upstream/master'
79de25d 
* upstream/master: (54 commits)
  Add notes about debug with libstdc++ symbols (apache#13533)
  add cpp example inception to nightly test (apache#13534)
  Fix exception handling api doc (apache#13519)
  fix link for gluon model zoo (apache#13583)
  ONNX import/export: Size (apache#13112)
  Update MXNetTutorialTemplate.ipynb (apache#13568)
  fix the situation where idx didn't align with rec (apache#13550)
  Fix use-before-assignment in convert_dot (apache#13511)
  License update  (apache#13565)
  Update version to v1.5.0 including clojure package (apache#13566)
  Fix flaky test test_random:test_randint_generator (apache#13498)
  Add workspace cleaning after job finished (apache#13490)
  Adding test for softmaxoutput (apache#13116)
  apache#13441 [Clojure] Add Spec Validations for the Random namespace (apache#13523)
  Revert "Bumped minor version from 1.4.0 to 1.5.0 on master, updated License file" (apache#13558)
  Chi_square_check for discrete distribution fix (apache#13543)
  Updated docs for randint operator (apache#13541)
  Simplifications and some fun stuff for the MNIST Gluon tutorial (apache#13094)
  Fix apache#13521 (apache#13537)
  Add a retry to qemu_provision (apache#13551)
  ...
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@szha szha szha approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Gluon DataLoader cannot release the processes in the pool
2 participants
@zhreshold @szha