[Cluster launcher] Add friendly warning for missing boto3 and googleapiclient imports

[architkulkarni]

Why are these changes needed?

Adds a friendlier warning when required packages for the cluster launcher are missing. boto3 in the case of AWS, and googleapiclient for GCP

Related issue number

Closes #39941

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[architkulkarni]

@krfricke

https://github.com/architkulkarni/ray/blob/1fd210222b79fca568bc341f1cd915f842f998ea/python/setup.py#L235-L237

# If you're adding dependencies for ray extras, please
# also update the matching section of requirements/requirements.txt
# in this directory

It seems the file structure has changed. I couldn't find the relevant requirements.txt file for ray[default], do you know where I should add it?

[architkulkarni]

Oh looks like it's just python/requirements.txt not requirements/requirements.txt?

No, that one doesn't have ray[default] either

[architkulkarni]

@krfricke

https://github.com/architkulkarni/ray/blob/1fd210222b79fca568bc341f1cd915f842f998ea/python/setup.py#L235-L237

# If you're adding dependencies for ray extras, please
# also update the matching section of requirements/requirements.txt
# in this directory

It seems the file structure has changed. I couldn't find the relevant requirements.txt file for ray[default], do you know where I should add it?

@can-anyscale do you happen to know where to add it?

[can-anyscale]

@architkulkarni it's that python/requirements.txt file; you can just add a new comment to create a session of [default] and add boto3 under it

you might need to update the python/requirements_compiled.txt (auto-generated code) as well (https://www.notion.so/anyscale-hq/OSS-Python-dependency-management-f32633b0018c423f927727807ea9da08)

[architkulkarni]

@architkulkarni it's that python/requirements.txt file; you can just add a new comment to create a session of [default] and add boto3 under it

you might need to update the python/requirements_compiled.txt (auto-generated code) as well (https://www.notion.so/anyscale-hq/OSS-Python-dependency-management-f32633b0018c423f927727807ea9da08)

@can-anyscale I don't see the pip-compile dependencies job described in https://www.notion.so/anyscale-hq/OSS-Python-dependency-management-f32633b0018c423f927727807ea9da08. Does that mean I don't need to update python/requirements_compiled.txt?

[can-anyscale]

Hi, normally python/requirements.txt requires an update (documentation is wrong?), but boto3 is already in the compiled list so in this particular case you don't need to update.

[architkulkarni]

Ahh makes sense, thanks!

[architkulkarni]

@edoakes do you mind reviewing this PR as codeowner?

[edoakes]

@architkulkarni not sure if this has previously been discussed but I'm not sure if we want to do this; this is essentially making the AWS cluster launcher a supported part of the default installation, and opting folks into requiring this dependency even if they're running on other platforms (or not using the cluster launcher). I believe the current "policy" is that the cluster launchers are community-maintained, optional extensions to Ray.

I think the better solution would be to have a separate install target for each cluster launcher variant, like ray["aws"], ray["gcp"], etc. (though didn't we have that at some point?).

[architkulkarni]

Discussed offline, we'll just add a better import warning. @edoakes please take another look

[hongchaodeng]

Nice!
LGTM