[CI] Fix Windows Wheel flaky build caused by ', using heredoc in refreshenv.

[cadedaniel]

Description

See #29878 and #29717 for context.

TL;DR there is a script which exports environment variables present in powershell to bash, but it fails to handle cases where environment variable values contain single quotes. This causes the Windows Wheels build to fail whenever an environment variable value contains a single quote, particularly when the commit message has a single quote (common!). I fixed it by exporting the variables using bash heredocs which are more robust against arbitrary input values.

Testing

Powershell code to test:

#Import-Module "$env:ChocolateyInstall\helpers\chocolateyProfile.psm1"
#Update-SessionEnvironment
# Round brackets in variable names cause problems with bash
Get-ChildItem env:* | %{
  if (!($_.Name.Contains('('))) {
    $value = $_.Value
    if ($_.Name -eq 'PATH') {
      $value = $value -replace ';',':'
    }
    # Write-Output ("export " + $_.Name + "='" + $value + "'")
    Write-Output ("export " + $_.Name + "=$`(cat <<- 'REFRESHENV_EOF'`n" + $value + "`nREFRESHENV_EOF`)")
  }
}

Test runner (COMPLEX_ENV_VAR_1 has single quotes):

#!/usr/bin/env bash

pwsh=$(which pwsh)

export COMPLEX_ENV_VAR_1=$(cat <<- "EOF"
[Jobs] Deflake jobs sdk test (#29707)
--
Wait for all nodes to come up before running the unit test to deflake the unit test. It submits a bunch of jobs and checks that at least one was run on a worker node. However if the jobs are submitted too soon, its' possible that only the head node has come up, so 'theyll all be scheduled on the head node. The fix is to only submit the jobs after all the nodes are up.
Also adds some logs to improve debuggability.

Related issue number
Closes #29006
EOF)

env -i COMPLEX_ENV_VAR_1="$COMPLEX_ENV_VAR_1" $pwsh ps.ps | sed 's/export PATH/export FAKE_PATH/g' > refreshenv.sh
echo 'env' >> refreshenv.sh

env -i bash refreshenv.sh

Output:

$ ./test.sh
FAKE_PATH=/usr/local/microsoft/powershell/7:
COMPLEX_ENV_VAR_1=[Jobs] Deflake jobs sdk test (#29707)
--
Wait for all nodes to come up before running the unit test to deflake the unit test. It submits a bunch of jobs and checks that at least one was run on a worker node. However if the jobs are submitted too soon, its' possible that only the head node has come up, so 'theyll all be scheduled on the head node. The fix is to only submit the jobs after all the nodes are up.
Also adds some logs to improve debuggability.

Related issue number
Closes #29006
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
PSModulePath=/tmp/71636ff1-42ce-40a1-9c08-d4e9ae135fad/.local/share/powershell/Modules:/usr/local/share/powershell/Modules:/usr/local/microsoft/powershell/7/Modules
PWD=/Users/cade/dev/cadelabs/ray-powershell-fix
SHLVL=1
_=/usr/bin/env

It worked! Without the heredoc, it gets borked:

$ ./test.sh
refreshenv.sh: line 3: export: `up,': not a valid identifier
refreshenv.sh: line 3: export: `theyll all be scheduled on the head node. The fix is to only submit the jobs after all the nodes are up.
Also adds some logs to improve debuggability.

Related issue number
Closes #29006': not a valid identifier
FAKE_PATH=/usr/local/microsoft/powershell/7:
COMPLEX_ENV_VAR_1=[Jobs] Deflake jobs sdk test (#29707)
--
Wait for all nodes to come up before running the unit test to deflake the unit test. It submits a bunch of jobs and checks that at least one was run on a worker node. However if the jobs are submitted too soon, its
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
PSModulePath=/tmp/81c31618-cb53-4d44-b01c-fd622dba6734/.local/share/powershell/Modules:/usr/local/share/powershell/Modules:/usr/local/microsoft/powershell/7/Modules
PWD=/Users/cade/dev/cadelabs/ray-powershell-fix
SHLVL=1
_=/usr/bin/env

[jjyao]

Could you add a single quote to the commit message to make sure it works :)

[cadedaniel]

haha, sure :)

[cadedaniel]

@jjyao my edit will go into the merge commit, right?

[cadedaniel]

@jjyao merge please

[cadedaniel]

🤔🤔🤔🤔

+ source /c/Users/ContainerAdministrator/AppData/Local/Temp/refreshenv.sh
/c/Users/ContainerAdministrator/AppData/Local/Temp/refreshenv.sh: line 6: warning: here-document at line 4 delimited by end-of-file (wanted `REFRESHENV_EOF')
+++ cat
++ export BUILDKITE=true
++ BUILDKITE=true
/c/Users/ContainerAdministrator/AppData/Local/Temp/refreshenv.sh: line 9: warning: here-document at line 7 delimited by end-of-file (wanted `REFRESHENV_EOF')
+++ cat
++ export 'ProgramW6432=C:\Program Files'
++ ProgramW6432='C:\Program Files'
/c/Users/ContainerAdministrator/AppData/Local/Temp/refreshenv.sh: line 12: warning: here-document at line 10 delimited by end-of-file (wanted `REFRESHENV_EOF')
+++ cat
++ export 'ChocolateyInstall=C:\ProgramData\chocolatey'
++ ChocolateyInstall='C:\ProgramData\chocolatey'
/c/Users/ContainerAdministrator/AppData/Local/Temp/refreshenv.sh: line 15: warning: here-document at line 13 delimited by end-of-file (wanted `REFRESHENV_EOF')