RFC: metric name formatting

[olix0r]

By default, Finagle clients format metrics as follows:

clnt/<label>/<metric>.<sfx>
srv/<label>/<metric>.<sfx>

Router stats don't exactly fit into the model--for instance the bindcache stats are currently unscoped (and therefore lossy/confusing when multiple routers exist). I'd like to propose a new naming scheme that more closely reflects reality and carves out space for furhther expansion:

rt/<router-label>/bindcache/...
rt/<router-label>/dst/path/<unbound-dst-path>/...
rt/<router-label>/dst/id/<bound-dst-id>/...
rt/<router-label>/srv/<server-ip>/<server-port/...
rt/<router-label>/...

This allows us to potentially expose multiple scopes of client stats (e.g. logical sr vs cluster sr -- logical is what you monitor, cluster is how you compare code versions).

Later, various stats scopes should be controllable should be controllable via configuration parameter (because for any large site, there may be many many unbound-dst-path values).

In order to complete this, the following things must happen:

• we need to craft stats scopes during router initialization
• the admin page must be updated to map client and server stat names.

Note: It is technically feasible to preserve the existing stat name structure, however to do so incurs complexity on both initialization and consumers for marginal value. For example, we could try to construct names like:

clnt/<router-label>/<bound-dst-id>/...
srv/<router-label>/<ip>/<port>/...
rt/<router-label>/bindcache
rt/<router-label/...

Regardless of implementation details, I think it's preferable to establish a single naming hierarchy for router stats. It also happens to be the case that it's much simpler to implement hierarchical stats scoping in router initialization.

Namer and Tracer stats should be scoped by namer/<label> and tracer/<label>, respectively.

This change will render some parts of TwitterServer's Admin page obsolete, but we intend to replace TwitterServer in favor of a config-driven admin page anyway. I don't believe that this change breaks any other APIs that we care about.

[siggy]

This sounds good. I think we should complete #16 prior to this work. Specifically, booting our own admin server, and serving our own metrics.json endpoint.

[olix0r]

@siggy I'd like to move forward with this independently of #16, since we need stats receivers to become sane to complete #6. I don't think that there are actually any dependencies we care about between these issues, so they should be able to happen in more-or-less parallel.