-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
failed to save outputs: interface conversion: error is *exec.Error, not *exec.ExitError #1207
Comments
workflow-controller log:
|
I just build from master ( plus a unrelated tweak ) and can confirm this behavior - except I get it every run |
@wadeholler thanks. it fails 80% of times for me. I think we upgraded the k8s cluster version and starts to see this. |
@wadeholler there had been a bug on master branch, I fixed it with the PR#1213. Try to delete |
That helped the stated problem above but now submodule support is broken:
|
my previous reply was for repos that had a submodule reference but no .gitmodules file. the new argoexec updates that force a submodule update caused this issue. unrelated to the above. All is well now. cheers |
I'm pretty sure I had fixed a |
- only attempt to list/join channels if postMessage call fails with not_in_channel - respect rate limit 429 responses when iterating through paginated conversations.list result fixes argoproj#1206 Signed-off-by: Robert King <[email protected]>
Is this a BUG REPORT or FEATURE REQUEST?:
BUG REPORT
What happened:
Seeing this error msg every so often and completely random. It does not happen specifically to a task. I get this for a task and the next run my task works fine.
The task runs fine and I can see the output but If the next task depends on this task it won't go to the next task.
What you expected to happen:
I did not used to see this and start to see that at some point of time when I installed kubeflowpipline and ran a task. However I remove/redeploy argo again but still see the error every so often.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
Other debugging information (if applicable):
argo get argo-gpu-s3-copy-4qzp8
Name: argo-gpu-s3-copy-4qzp8
Namespace: development
ServiceAccount: argo
Status: Error
Created: Sat Feb 02 19:10:05 +0000 (3 minutes ago)
Started: Sat Feb 02 19:10:05 +0000 (3 minutes ago)
Finished: Sat Feb 02 19:10:21 +0000 (3 minutes ago)
Duration: 16 seconds
Parameters:
s3-path: Shared_data/OULU/small_frames_npy
local-path: test2
bucker-name: onfido-mlplatform-in
node-selector: m4.xlarge
STEP PODNAME DURATION MESSAGE
⚠ argo-gpu-s3-copy-4qzp8
└-⚠ list-chunk argo-gpu-s3-copy-4qzp8-3040831338 16s failed to save outputs: interface conversion: error is *exec.Error, not *exec.ExitError
The text was updated successfully, but these errors were encountered: