Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

adding stacktrace in Jenkinsfile_utils.groovy #17065

Merged
merged 1 commit into from
Dec 16, 2019

Conversation

access2rohit
Copy link
Contributor

@access2rohit access2rohit commented Dec 13, 2019

Description

to inspect cause of Python2 failure in CI (Unix CPU)
To identify root cause of : #16995

Upon Inspecting different timeout on Unix CPU in the following runs:
http:https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16965/12/pipeline/296 Py3 CPU Debug
http:https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16971/8/pipeline/294 Py2 CPU
http:https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17018/5/pipeline/294 Py2 CPU
http:https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/master/1387/pipeline/47 CPU MKLDNN / CPU_MKLDNN_MKL (build)

http:https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/PR-16885/runs/9/nodes/294/steps/786/log/?start=0 Py2 CPU
http:https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/master/runs/1371/nodes/294/steps/747/log/?start=0 Py2 CPU
http:https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/master/runs/1336/nodes/294/steps/796/log/?start=0 Py2 CPU
http:https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/master/runs/1337/nodes/294/steps/784/log/?start=0 Py2 CPU

All the tests passed when looked into the logs of Jenkin's run but it fails with message:

End of Pipeline
Timeout has been exceeded
Finished: ABORTED

you can see many files when you grep for Cancelling nested steps due to timeout, but all of them have just one line : caught "Exception name"

This PR prints the stacktrace of the exception to help identify the root cause of CI timeouts in jenkins

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@access2rohit
Copy link
Contributor Author

access2rohit commented Dec 13, 2019

@anirudh2290 @marcoabreu Can you review ?

@anirudh2290
Copy link
Member

were you able to test this somewhere ?

@access2rohit
Copy link
Contributor Author

access2rohit commented Dec 13, 2019

No I tested this code in a different file for stacktrace. I don't know how to. Let me know if you do

@anirudh2290
Copy link
Member

Apologies, i dont know how it edited your comment .

No I tested this code in a different file for stacktrace. I don't know how to. Let me know if you do

Okay, thats what I was looking for. Testing the line of code if it works in general. To test in this specific code, you can throw an exception from throw block and see if it prints stacktrace in a dev environment.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants