[Doc] [Job] Add notes about where Ray Job entrypoint runs and how to specify it #41319
Conversation
Signed-off-by: Archit Kulkarni <[email protected]>
architkulkarni
requested review from
maxpumperla,
pcmoritz,
kevin85421 and
a team
as code owners
doc/source/cluster/running-applications/job-submission/quickstart.rst
Outdated
Show resolved
Hide resolved
| @@ -111,12 +111,19 @@ Make sure to specify the path to the working directory in the ``--working-dir`` | |||
| # Job 'raysubmit_inB2ViQuE29aZRJ5' succeeded | |||
| # ------------------------------------------ | |||
|
|
|||
| This command will run the script on the Ray Cluster and wait until the job has finished. Note that it also streams the stdout of the job back to the client (``hello world`` in this case). Ray will also make the contents of the directory passed as `--working-dir` available to the Ray job by downloading the directory to all nodes in your cluster. | |||
| This command will run the entrypoint script on the Ray Cluster's head node and wait until the job has finished. Note that it also streams the stdout of the job back to the client (``hello world`` in this case). Ray will also make the contents of the directory passed as `--working-dir` available to the Ray job by downloading the directory to all nodes in your cluster. | |||
There was a problem hiding this comment.
It not only streams STDOUT but also STDERR. You can do a simple experiment:
# example.py
import sys
# Print a message to stdout
print("This is a message to stdout")
# Print a message to stderr
print("This is an error message to stderr", file=sys.stderr)
There was a problem hiding this comment.
Good point. I just want to convey that it would stream whatever is normally output to the terminal when you run a command in your local terminal. Should I say "streams the stdout and stderr" or "streams the output"?
There was a problem hiding this comment.
Maybe the former? A lot of commands/tools do not stream stderr by default.
There was a problem hiding this comment.
Updated to "streams the output of the entrypoint script", which should be clear. 61d65ff
There was a problem hiding this comment.
Oh sorry, didn't see your message. Updated to "streams the stdout and stderr" ab7950c
|
|
||
| .. note:: | ||
|
|
||
| The double dash (`--`) separates the arguments for the entrypoint command (e.g. `python script.py --arg1=val1`) from the arguments to `ray job submit`. | ||
|
|
||
| .. note:: | ||
|
|
||
| By default the entrypoint script is run on the head node. To override this, specify any of the arguments |
There was a problem hiding this comment.
"entrypoint script is run on the head node" => Do you mean the driver process would be running on the head node by default?
There was a problem hiding this comment.
Yes, we say entrypoint script here to convey that it is running whatever the user specifies as entrypoint. Typically this is a script that starts a Ray driver process (ray.init()), but it could also be any command at all, like echo hello && pip install something. It technically doesn't have to involveRay
There was a problem hiding this comment.
Short answer, yes, the driver is running on the head node by default
…art.rst Co-authored-by: Kai-Hsun Chen <[email protected]> Signed-off-by: Archit Kulkarni <[email protected]>
Signed-off-by: Archit Kulkarni <[email protected]>
…i/ray into docs-job-head-node Signed-off-by: Archit Kulkarni <[email protected]>
Signed-off-by: Archit Kulkarni <[email protected]>
Signed-off-by: Archit Kulkarni <[email protected]>
Quick follow to #41319 --------- Signed-off-by: angelinalg <[email protected]> Co-authored-by: Archit Kulkarni <[email protected]>
…specify it (ray-project#41319) There is recurring user confusion about where the job entrypoint script runs and how to make it run on a worker node. This PR adds the missing information to the doc in relevant places in the tutorials, and includes it in the FAQ. --------- Signed-off-by: Archit Kulkarni <[email protected]> Signed-off-by: Archit Kulkarni <[email protected]> Co-authored-by: Kai-Hsun Chen <[email protected]>
…ct#41342) Quick follow to ray-project#41319 --------- Signed-off-by: angelinalg <[email protected]> Co-authored-by: Archit Kulkarni <[email protected]>
Why are these changes needed?
There is recurring user confusion about where the job entrypoint script runs and how to make it run on a worker node.
This PR adds the missing information to the doc in relevant places in the tutorials, and includes it in the FAQ.
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.