[ci] migrate remaining non-gpu ml tests to civ2

[can-anyscale]

• Install hdfs into the ml.build image as well, so we can reuse the same images for a lot of test settings
• Use bash as a default shell for these test images

Test:

• CI

[can-anyscale]

this is the only new line here

[can-anyscale]

this is required to use oss-ci-base_build for testing

[aslonnie]

why? what is the error?

rayci already sets shell to -elic for the docker plugin?

[can-anyscale]

yeah -elic is only provided during docker run; this is docker build; the oss_ci_base_build is already have pip installed (through ci/env/install-dependencies.sh i assume), but without this the docker build process will say pip not found (assume pip is loaded through .bashrc)

[can-anyscale]

the error: https://buildkite.com/ray-project/premerge/builds/9322#018b4e69-c8d5-4bdb-ad01-ac0a9910630c/6-99

[aslonnie]

in the build, it has !/bin/bash shebang. I think the only thing you want is -i, which adds anaconda's pip into PATH, you do not want to enable -c and -e by default though. that would be quite surprising result for users.

it might be easier and cleaner to add $HOME/anaconda3/bin/ into PATH (export PATH="$HOME/anaconda3/bin:$PATH" at the beginning of the script.

[can-anyscale]

Keeping only -i sounds good, I want to load the .bashrc instead of only getting pip, there are quite a few places setting things up in .bashrc (https://github.com/ray-project/ray/blob/master/.buildkite/pipeline.ml.yml#L367).

A few things at put in root/.bashrc, others are in ~/.bashrc, so -i will be a safer option to set all things up.

[can-anyscale]

in this image, the default shell is sh not bash

[can-anyscale]

@aslonnie I still need to use at least bash -ic for the shell to loads .bash_rc; if there is a better way to do this let me know