[runtime env] Change pip_check default from True to False

[architkulkarni]

Why are these changes needed?

@SongGuyang @Catch-Bull @edoakes I know we discussed this earlier, but after thinking about it some more I think a more reasonable default is for pip check to be False by default. My guess is that a lot of users (including myself) work inside an environment where python -m pip check fails, but the environment doesn't cause them any problems otherwise. So a lot of users will hit an error when trying a simple runtime_env pip example, and possibly give up. Another less important piece of evidence is that we had to set pip_check = False to make some CI tests pass in the original PR.

This also matches the default behavior of pip which allows this situation to occur in the first place: pip install doesn't error when there's a dependency conflict; rather the command succeeds, the package is installed and usable, and it prints a warning (which is confusingly titled "ERROR")

Example:

Installing collected packages: sphinx
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
sphinx-tabs 2.1.0 requires sphinx<4,>=2, but you have sphinx 4.3.2 which is incompatible.
Successfully installed sphinx-4.3.2
❯ echo $?
0

Related issue number

Closes #23291

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[architkulkarni]

@edoakes @SongGuyang @Catch-Bull What do you think about this?

[SongGuyang]

Make sense. We should avoid to break users' configuration. Actually, a lot of developers don't know the command ｀pip check`. Only one concern is that can we ensure the dependencies of ray are not broken by users' packages? We already check ray version now, but I don't know if the dependencies could be covered. @Catch-Bull Can you also check this?

[Catch-Bull]

Make sense. We should avoid to break users' configuration. Actually, a lot of developers don't know the command ｀pip check`. Only one concern is that can we ensure the dependencies of ray are not broken by users' packages? We already check ray version now, but I don't know if the dependencies could be covered. @Catch-Bull Can you also check this?

As far as I know, pip doesn't generate new conflicts to resolve old conflicts yet, which means that even if pip modifies ray's dependencies, there will be no conflict.

[Catch-Bull]

That makes sense to me.

But I think users should also be informed that the current configuration is at risk, maybe we should open a new PR to implement this, I think there have two ways to implement this.

1. log warning information when RuntimeEnv construct. The advantage is that the implementation is simple, the disadvantage is that the construction of RuntimeEnv is too frequent, and too many logs may be printed.
2. log warning information when PipManager.run been called on runtime_env_agent. The advantage is that it has the opportunity to tell the user what specific conflicts exist. The disadvantage is that it is more complicated to implement, and it is also necessary to think about how to reveal the stdout of the driver or ray.client process.

[architkulkarni]

Warning the user in a followup PR sounds good to me. Thanks for the two ideas, both seem reasonable--maybe we can add an environment variable flag to disable pip warnings as well.