[Serve] [Docs] Update the "Monitoring Ray Serve" Page #27777

shrekris-anyscale · 2022-08-11T06:24:35Z

Why are these changes needed?

The "Monitoring Ray Serve" page explains how to inspect your Ray Serve applications. This change updates the page to remove outdated metrics that Serve no longer exposes and to upgrade code samples to use 2.0 APIs. It also improves the content's readability and organization.

Link to updated "Monitoring Ray Serve" page: https://ray--27777.org.readthedocs.build/en/27777/serve/monitoring.html

Related issue number

Closes #27720.

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
  - This change adds assertions to the documentation code, so they're tested in CI.

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

doc/source/serve/monitoring.md

edoakes · 2022-08-11T17:00:24Z

@shrekris-anyscale FYI you have a merge conflict here

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

edoakes

Thanks for adding the serve.run changes, those will be very helpful.

edoakes · 2022-08-11T20:07:29Z

doc/source/serve/monitoring.md

+This includes details such as:
+* the number of deployment replicas currently running
+* logs for your Serve controller, deployment replicas, and HTTP proxies
+* the Ray nodes (i.e. machines) running in your Ray cluster


period to end the last bullet point

edoakes · 2022-08-11T20:07:58Z

doc/source/serve/monitoring.md

+For example, if you're running Ray Serve on a local Ray cluster, you can access the dashboard by going to this address in your browser:
+
+```
+https://localhost:8265
+```


Suggested change

For example, if you're running Ray Serve on a local Ray cluster, you can access the dashboard by going to this address in your browser:

```

https://localhost:8265

```

For example, if you're running Ray Serve locally, you can access the dashboard by going to `https://localhost:8265` in your browser.

save some page real estate

edoakes · 2022-08-11T20:08:34Z

doc/source/serve/monitoring.md

+We can view important information about our application here.
+For example, we can inspect our deployment replicas by navigating to the Ray dashboard's "Actors" tab while our Serve application is running:


the previous paragraph used "you" and now you're using "we," stick with one

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

shrekris-anyscale · 2022-08-11T20:32:12Z

@edoakes Thanks for the review! I addressed your comments.

dmatrix

With those changes, it should be good to go.

Just a general comment. If possible for 2.1, we should consider providing dashboards for this; there is too much boiler code they have to write. But at least we are collecting stats; matter of accessing and presenting easily.

dmatrix · 2022-08-11T21:37:13Z

doc/source/serve/monitoring.md

-This section should help you understand how to debug and monitor your Serve application.
-There are three main ways to do this:
-Using the Ray Dashboard, using Ray logging, and using built-in Ray Serve metrics.
+This section should help you debug and monitor your Serve applications by:


This section helps you debug and monitor your Serve applications by:

dmatrix · 2022-08-11T21:45:01Z

doc/source/serve/monitoring.md

+## Ray logging
+
+You can use Ray logging to understand system-level behavior and to surface application-level details during runtime.
+


We repeatedly begin a sentence with "You can use XXX..." Perhaps change the pattern: may be use an infinitive phrase or participle phrase :

To comprehend system-level behavior and surface application-level details during runtime, use Ray logging.

dmatrix · 2022-08-11T21:46:32Z

doc/source/serve/monitoring.md

 ```

-Running this code block, we first get some log messages from the controller saying that a new replica of the deployment is being created:
+We can run this deployment using the `serve run` CLI command:


Run this deployment using .....

dmatrix · 2022-08-11T21:47:34Z

doc/source/serve/monitoring.md

 ```

-Then when we query the deployment, we get both a default access log as well as our custom `"Hello world!"` message.
+`serve run` prints out a few log messages immediately. Note that a few of these messages start with identifiers such as


"out" may be redundant. This without "out" reads equally well.

prints a few log messages ....

dmatrix · 2022-08-11T22:03:03Z

doc/source/serve/monitoring.md

@@ -82,18 +111,21 @@ class Silenced:
 logger.setLevel(logging.ERROR)
 ```

-This will prevent the replica INFO-level logs from being written to STDOUT or to files on disk.
+This controls which logs are written to STDOUT or files on disk.
 You can also use your own custom logger, in which case you'll need to configure the behavior to write to STDOUT/STDERR, files on disk, or both.



Apart from standard Python logger, Serve supports custom logging. Custom logging via configuration can control the behavior of what messages are written to STDOUT/STDERR, files on disk, or both.

dmatrix · 2022-08-11T22:07:40Z

doc/source/serve/monitoring.md

-First, install Loki and Promtail using the instructions on <https://grafana.com>.
-It will be convenient to save the Loki and Promtail executables in the same directory, and to navigate to this directory in your terminal before beginning this walkthrough.
+You can explore and filter your logs using [Loki](https://grafana.com/oss/loki/).
+Setup and configuration is straightforward on Kubernetes, but as a tutorial, let's set up Loki manually.



Is "setup and configuration" considered a singular compound subject (like ham and cheese is great for breakfast) in this context? If not perhaps change to

"Setup and configuration are straightforward on Kubernetes."

dmatrix · 2022-08-11T22:14:35Z

doc/source/serve/monitoring.md

@@ -163,11 +198,14 @@ You should see something similar to the following:
 :align: center


passward "admin."

dmatrix · 2022-08-11T22:17:21Z

doc/source/serve/monitoring.md

 ```

-The requests will loop and can be canceled with `Ctrl-C`. 
+The requests will loop and can be canceled with `ctrl-c`. 



The requests will loop until canceled with ctrl-c.

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

shrekris-anyscale · 2022-08-11T22:50:21Z

Thanks for the review @dmatrix! I've addressed all your comments.

Just a general comment. If possible for 2.1, we should consider providing dashboards for this; there is too much boiler code they have to write. But at least we are collecting stats; matter of accessing and presenting easily.

Yeah that's a good suggestion. I think that would improve Ray Serve's observability.

) The "Monitoring Ray Serve" page explains how to inspect your Ray Serve applications. This change updates the page to remove outdated metrics that Serve no longer exposes and to upgrade code samples to use 2.0 APIs. It also improves the content's readability and organization. Link to updated "Monitoring Ray Serve" page: https://ray--27777.org.readthedocs.build/en/27777/serve/monitoring.html

) The "Monitoring Ray Serve" page explains how to inspect your Ray Serve applications. This change updates the page to remove outdated metrics that Serve no longer exposes and to upgrade code samples to use 2.0 APIs. It also improves the content's readability and organization. Link to updated "Monitoring Ray Serve" page: https://ray--27777.org.readthedocs.build/en/27777/serve/monitoring.html Signed-off-by: Stefan van der Kleij <[email protected]>

) The "Monitoring Ray Serve" page explains how to inspect your Ray Serve applications. This change updates the page to remove outdated metrics that Serve no longer exposes and to upgrade code samples to use 2.0 APIs. It also improves the content's readability and organization. Link to updated "Monitoring Ray Serve" page: https://ray--27777.org.readthedocs.build/en/27777/serve/monitoring.html

) The "Monitoring Ray Serve" page explains how to inspect your Ray Serve applications. This change updates the page to remove outdated metrics that Serve no longer exposes and to upgrade code samples to use 2.0 APIs. It also improves the content's readability and organization. Link to updated "Monitoring Ray Serve" page: https://ray--27777.org.readthedocs.build/en/27777/serve/monitoring.html Signed-off-by: Artur Niederfahrenhorst <[email protected]>

shrekris-anyscale added 11 commits August 10, 2022 16:22

Update metrics list in documentation

252993a

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Move code out of legacy folder

11dc6b0

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Move remaining code from legacy location

8cd69ab

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Add test to monitoring.py

8c83287

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Move code files to new folder and revise intro

785e13e

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Capitalize dashboard

e714773

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Update copy

b47517e

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Rewrite 'Ray dashboard' section

e7eb94e

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Revise the Loki section

64cc4ec

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Revise 'Ray logging' section

e76e2c9

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Update 'Built-in Ray Serve metrics' section

9603e91

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

shrekris-anyscale assigned edoakes Aug 11, 2022

shrekris-anyscale requested review from edoakes, simon-mo, jiaodong and sihanwang41 as code owners August 11, 2022 06:24

shrekris-anyscale assigned simon-mo Aug 11, 2022

shrekris-anyscale requested a review from a team as a code owner August 11, 2022 06:24

shrekris-anyscale added 3 commits August 10, 2022 23:25

Merge branch 'master' of github.com:ray-project/ray into metrics_update

8de9718

Fix typo

ed61cb2

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

Refine based on rendered output

ffb58a9

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

shrekris-anyscale commented Aug 11, 2022

View reviewed changes

doc/source/serve/monitoring.md Outdated Show resolved Hide resolved

shrekris-anyscale assigned dmatrix and shrekris-anyscale and unassigned dmatrix, edoakes and simon-mo Aug 11, 2022

shrekris-anyscale added 2 commits August 11, 2022 10:24

Merge branch 'master' of github.com:ray-project/ray into metrics_update

bae4081

Revise sentence

ba5ae3d

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

shrekris-anyscale assigned dmatrix, edoakes and simon-mo Aug 11, 2022

edoakes reviewed Aug 11, 2022

View reviewed changes

edoakes added the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Aug 11, 2022

Address @edoakes's comments

6640ea2

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

shrekris-anyscale removed the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Aug 11, 2022

edoakes approved these changes Aug 11, 2022

View reviewed changes

edoakes added copyediting-required and removed copyediting-required labels Aug 11, 2022

dmatrix approved these changes Aug 11, 2022

View reviewed changes

Implement @dmatrix's suggestions

d55bac4

Signed-off-by: Shreyas Krishnaswamy <[email protected]>

edoakes merged commit e15960e into ray-project:master Aug 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Serve] [Docs] Update the "Monitoring Ray Serve" Page #27777

[Serve] [Docs] Update the "Monitoring Ray Serve" Page #27777

shrekris-anyscale commented Aug 11, 2022 •

edited

Loading

edoakes commented Aug 11, 2022

edoakes left a comment

edoakes Aug 11, 2022

edoakes Aug 11, 2022

edoakes Aug 11, 2022

shrekris-anyscale commented Aug 11, 2022

dmatrix left a comment

dmatrix Aug 11, 2022

dmatrix Aug 11, 2022

dmatrix Aug 11, 2022

dmatrix Aug 11, 2022

dmatrix Aug 11, 2022

dmatrix Aug 11, 2022

dmatrix Aug 11, 2022

dmatrix Aug 11, 2022

shrekris-anyscale commented Aug 11, 2022

		We can view important information about our application here.
		For example, we can inspect our deployment replicas by navigating to the Ray dashboard's "Actors" tab while our Serve application is running:

		## Ray logging

		You can use Ray logging to understand system-level behavior and to surface application-level details during runtime.

		@@ -163,11 +198,14 @@ You should see something similar to the following:
		:align: center

[Serve] [Docs] Update the "Monitoring Ray Serve" Page #27777

[Serve] [Docs] Update the "Monitoring Ray Serve" Page #27777

Conversation

shrekris-anyscale commented Aug 11, 2022 • edited Loading

Why are these changes needed?

Related issue number

Checks

edoakes commented Aug 11, 2022

edoakes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shrekris-anyscale commented Aug 11, 2022

dmatrix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shrekris-anyscale commented Aug 11, 2022

shrekris-anyscale commented Aug 11, 2022 •

edited

Loading