Command being replaced into the workflow: Golang fundamental flaw bug - again

[everton-nasc]

Summary

I’m facing a quite weird issue when I’m trying to suppress the stderr from the phpunit tests executions and not sure if the workflow command translation is replacing and send the command to the container.

Issue
We’re appending 2> /dev/null at the end of the phpunit binary (e.g: $phpunit 2>dev/null) - this is a workaround to avoid the stderr payloads being printed during the test execution.
Noticed the command at the Summary tab (name value) has changed from 2> /dev/null to 2u003e/dev/null
The images are showing the modified command along with the container log output:

Console output

ArgoWF Summary output

Diagnostics

The phpunit is failing due the incorrect command parameter received from the workflow (command).

Expected: phpunit --group unit 2> /dev/null
Found: phpunit --group unit 2\u300e/dev/null

Kubernetes provider: 1.18.15-gke.1100
Argo Workflow version: V3.0.0-rc3

- name: test-phpunit
        templateRef: 
          name: php
          template: main
        arguments:
          parameters:
            - name: repo
              value: "{{workflow.parameters.repo}}"
            - name: repo-full-name
              value: "{{workflow.parameters.repo-full-name}}"
            - name: revision
              value: "{{workflow.parameters.revision}}"
            - name: sha
              value: "{{workflow.parameters.sha}}"
            - name: command
              value: "{{item.command}}"
            - name: context
              value: "{{item.context}}"
            - name: tag
              value: "{{item.tag}}"
        withItems:
           - { tag: 'Unit-Test', command: './bin/phpunit --group unit 2> /dev/null', context: 'Unit Test' }
           - { tag: 'Integration-Test', command: './bin/phpunit --group integration 2> /dev/null',  context: 'Integration Test' }

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[alexec]

Can you confirm if this is a regression and if so which version it is regresses from.

[everton-nasc]

@alexec - Just double checked and I found the same behaviour on the version V3.0.0.0-rc2.

[alexec]

And did it work in v2.12.9?

[everton-nasc]

Yep @alexec, same error on version 2.12.9 😢
It seems the string (2>) is not being decoded properly.

[alexec]

Can you try v2.11.8?

[everton-nasc]

Hey @alexec,
I could't replicate the issue on v2.11.8. It seems to be an issues from the newest versions.
(also I having troubles with minikube port-forward to load the argo-ui, not related to the issue - something wrong with my local dev env).
Happy to help if you need further information.

[alexec]

By "couldn't replicate" do you mean it is not a problem is v2.11.8? Or something else?

[everton-nasc]

By "couldn't replicate" do you mean it is not a problem is v2.11.8? Or something else?

Yes.
Seems the decoding is working properly at 2.11.8 and the following versions are failing: 2.12.19, 3.0.0-rc2 and 3.0.0-rc3

[alexec]

Can I ask you to test v2.12.8? I think the revert in that version maybe re-introduced a bug.

[everton-nasc]

Note: I had a cache issues during my analysis and my previous analysis should be ignored.

Here's the results:
v2.12.8 --> PASSED

v3.0.0-rc1 --> FAILED

v3.0.0-rc2--> FAILED

v3.0.0-rc3--> FAILED

The issue still happening only on the latest version V3 releases.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: loops-
spec:
  entrypoint: loop-example
  templates:
  - name: loop-example
    steps:
    - - name: print-message
        template: whalesay
        arguments:
          parameters:
           - name: command
             value: 'echo test && {{item.command}}'
        withItems:              # invoke whalesay once for each item in parallel
        - {command: 'echo testing 1 && 2>/dev/null'}         # item 1
        - {command: 'echo testing 2 && 2>/dev/null'}

  - name: whalesay
    inputs:
      parameters:
      - name: command
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["{{inputs.parameters.command}}"]

Live object

apiVersion: v1
items:
- apiVersion: argoproj.io/v1alpha1
  kind: Workflow
  metadata:
    creationTimestamp: "2021-02-26T16:44:24Z"
    generateName: loops-
    generation: 4
    labels:
      workflows.argoproj.io/completed: "true"
      workflows.argoproj.io/phase: Succeeded
    managedFields:
    - apiVersion: argoproj.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:generateName: {}
        f:spec:
          .: {}
          f:arguments: {}
          f:entrypoint: {}
          f:serviceAccountName: {}
          f:templates: {}
        f:status: {}
      manager: argo
      operation: Update
      time: "2021-02-26T16:44:24Z"
    - apiVersion: argoproj.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:labels:
            .: {}
            f:workflows.argoproj.io/completed: {}
            f:workflows.argoproj.io/phase: {}
        f:status:
          f:conditions: {}
          f:finishedAt: {}
          f:nodes:
            .: {}
            f:loops-9c4p8:
              .: {}
              f:children: {}
              f:displayName: {}
              f:finishedAt: {}
              f:id: {}
              f:name: {}
              f:outboundNodes: {}
              f:phase: {}
              f:progress: {}
              f:resourcesDuration:
                .: {}
                f:cpu: {}
                f:memory: {}
              f:startedAt: {}
              f:templateName: {}
              f:templateScope: {}
              f:type: {}
            f:loops-9c4p8-764430937:
              .: {}
              f:boundaryID: {}
              f:children: {}
              f:displayName: {}
              f:finishedAt: {}
              f:id: {}
              f:name: {}
              f:phase: {}
              f:progress: {}
              f:resourcesDuration:
                .: {}
                f:cpu: {}
                f:memory: {}
              f:startedAt: {}
              f:templateName: {}
              f:templateScope: {}
              f:type: {}
            f:loops-9c4p8-3335239276:
              .: {}
              f:boundaryID: {}
              f:displayName: {}
              f:finishedAt: {}
              f:hostNodeName: {}
              f:id: {}
              f:inputs:
                .: {}
                f:parameters: {}
              f:name: {}
              f:outputs:
                .: {}
                f:exitCode: {}
              f:phase: {}
              f:progress: {}
              f:resourcesDuration:
                .: {}
                f:cpu: {}
                f:memory: {}
              f:startedAt: {}
              f:templateName: {}
              f:templateScope: {}
              f:type: {}
            f:loops-9c4p8-3687864728:
              .: {}
              f:boundaryID: {}
              f:displayName: {}
              f:finishedAt: {}
              f:hostNodeName: {}
              f:id: {}
              f:inputs:
                .: {}
                f:parameters: {}
              f:name: {}
              f:outputs:
                .: {}
                f:exitCode: {}
              f:phase: {}
              f:progress: {}
              f:resourcesDuration:
                .: {}
                f:cpu: {}
                f:memory: {}
              f:startedAt: {}
              f:templateName: {}
              f:templateScope: {}
              f:type: {}
          f:phase: {}
          f:progress: {}
          f:resourcesDuration:
            .: {}
            f:cpu: {}
            f:memory: {}
          f:startedAt: {}
      manager: workflow-controller
      operation: Update
      time: "2021-02-26T16:45:20Z"
    name: loops-9c4p8
    namespace: argo
    resourceVersion: "666"
    uid: db94f56c-11f2-4f5a-a215-e8e5e278bfa8
  spec:
    arguments: {}
    entrypoint: loop-example
    serviceAccountName: argo
    templates:
    - arguments: {}
      inputs: {}
      metadata: {}
      name: loop-example
      outputs: {}
      steps:
      - - arguments:
            parameters:
            - name: command
              value: echo test && {{item.command}}
          name: print-message
          template: whalesay
          withItems:
          - command: echo testing 1 && 2>/dev/null
          - command: echo testing 2 && 2>/dev/null
    - arguments: {}
      container:
        args:
        - '{{inputs.parameters.command}}'
        command:
        - cowsay
        image: docker/whalesay:latest
        name: ""
        resources: {}
      inputs:
        parameters:
        - name: command
      metadata: {}
      name: whalesay
      outputs: {}
  status:
    conditions:
    - status: "True"
      type: Completed
    finishedAt: "2021-02-26T16:45:20Z"
    nodes:
      loops-9c4p8:
        children:
        - loops-9c4p8-764430937
        displayName: loops-9c4p8
        finishedAt: "2021-02-26T16:45:20Z"
        id: loops-9c4p8
        name: loops-9c4p8
        outboundNodes:
        - loops-9c4p8-3335239276
        - loops-9c4p8-3687864728
        phase: Succeeded
        progress: 2/2
        resourcesDuration:
          cpu: 52
          memory: 52
        startedAt: "2021-02-26T16:44:24Z"
        templateName: loop-example
        templateScope: local/loops-9c4p8
        type: Steps
      loops-9c4p8-764430937:
        boundaryID: loops-9c4p8
        children:
        - loops-9c4p8-3335239276
        - loops-9c4p8-3687864728
        displayName: '[0]'
        finishedAt: "2021-02-26T16:45:20Z"
        id: loops-9c4p8-764430937
        name: loops-9c4p8[0]
        phase: Succeeded
        progress: 2/2
        resourcesDuration:
          cpu: 52
          memory: 52
        startedAt: "2021-02-26T16:44:24Z"
        templateName: loop-example
        templateScope: local/loops-9c4p8
        type: StepGroup
      loops-9c4p8-3335239276:
        boundaryID: loops-9c4p8
        displayName: print-message(0:command:echo testing 1 && 2>/dev/null)
        finishedAt: "2021-02-26T16:45:10Z"
        hostNodeName: minikube
        id: loops-9c4p8-3335239276
        inputs:
          parameters:
          - name: command
            value: echo test && echo testing 1 && 2>/dev/null
        name: loops-9c4p8[0].print-message(0:command:echo testing 1 && 2>/dev/null)
        outputs:
          exitCode: "0"
        phase: Succeeded
        progress: 1/1
        resourcesDuration:
          cpu: 27
          memory: 27
        startedAt: "2021-02-26T16:44:24Z"
        templateName: whalesay
        templateScope: local/loops-9c4p8
        type: Pod
      loops-9c4p8-3687864728:
        boundaryID: loops-9c4p8
        displayName: print-message(1:command:echo testing 2 && 2>/dev/null)
        finishedAt: "2021-02-26T16:45:10Z"
        hostNodeName: minikube
        id: loops-9c4p8-3687864728
        inputs:
          parameters:
          - name: command
            value: echo test && echo testing 2 && 2>/dev/null
        name: loops-9c4p8[0].print-message(1:command:echo testing 2 && 2>/dev/null)
        outputs:
          exitCode: "0"
        phase: Succeeded
        progress: 1/1
        resourcesDuration:
          cpu: 25
          memory: 25
        startedAt: "2021-02-26T16:44:24Z"
        templateName: whalesay
        templateScope: local/loops-9c4p8
        type: Pod
    phase: Succeeded
    progress: 2/2
    resourcesDuration:
      cpu: 52
      memory: 52
    startedAt: "2021-02-26T16:44:24Z"
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""