[Core] Ray c++ backend structured logging

[jjyao]

Why are these changes needed?

Introduced RAY_BACKEND_LOG_JSON=1 to control whether to emit backend c++ log in plain text or json.

To add contextual information to the log, use RAY_LOG(INFO).WithField("key1", "value1").WithField("key2", "value2") << "normal message" (This is the same as the existing RAY_EVENT)

Examples:

JSON format:

{"asctime":"2024-05-24 12:53:23,080","levelname":"I","message":"(raylet) Initializing NodeManager","filename":"node_manager.cc","lineno":287,"node_id":"99adb80d4e91c4765ef21f4964519d0743da3717bcfadbf80b0ede0a"}

TEXT format:

[2024-05-24 13:05:42,685 I 80577 16776052] (raylet) node_manager.cc:287: Initializing NodeManager node_id=b93cd25a3af0e1a178e0ceab636d2da14a0f094e0904ab0f09f5d373

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[rynewang]

can we use nlohmann/json to dump a json string instead of doing string concats and formats on our own?

[rynewang]

It's possible for user to provide a same key multiple times, resulting in a json with duplicated keys. It's technically legal but I don't think we want it. To avoid this, maybe we can do this:

1. in RayLog members, remove context_osstream_ and instead keep a map<str, str> fields_
2. in WithField, add to that fields_ (overwriting existing keys)
3. in ~RayLog, format the map in json or text mode, a la

#include <nlohmann/json.hpp>
#include "absl/strings/str_join.h"

std::string formatMap(const std::map<std::string, std::string>& map, bool jsonMode) {
    if (jsonMode) {
        nlohmann::json j(map);
        return j.dump();
    } else {
        return absl::StrJoin(map, " ", absl::PairFormatter("="));
    }
}

[rynewang]

This API requires all fields happen before any text strings, for which I am fine, though it may introduce some inconveniences when we have some optional outputs.

However it also only accepts strings, which seems to be a big deal, since it forces a base_id.Hex() call. What about:

template <typename T>
RayLog &RayLog::WithField(std::string_view key, const T &value) {
    std::stringstream ss;
    ss << value;
    // put ss.str() to field_ or context_osstream_...
}

[rynewang]

what about:

constexpr std::string_view

And, are these keys pre-defined? Can/Should users use custom keys?

[jjyao]

Users can provide custom keys. These are just common pre-defined keys that we will likely index.

[rynewang]

LGTM with nits