Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

service_creation/successes metric #870

Open
wmorgan opened this issue Dec 20, 2016 · 2 comments
Open

service_creation/successes metric #870

wmorgan opened this issue Dec 20, 2016 · 2 comments

Comments

@wmorgan
Copy link
Member

wmorgan commented Dec 20, 2016

For the purposes of detecting dead clusters (i.e. exist in service discovery, but every endpoint is down), it would be nice to have a service_creation/successes metrics that can be correlated with service_creation/failures. Currently, only the failures metric exists.

(Not 100% sure this is the right metric, but happy to discuss in the ticket.)
@adleong
Copy link
Member

adleong commented Dec 20, 2016

Taking a step back, the issue is that we want to be able to count requests that are delegated to a certain concrete id but that cluster cannot be connected to. In this case we might expect rt/<dst id>/requests and rt/<dst id>/failures to be incremented, but they are not. I'm guessing this is because those stats are incremented in the client stack but the request is aborted before then if no connections can be established at all.

Instead, the service_creation/failures stat is incremented which is less useful for counting how many individual requests with that concrete id failed.

@adleong
Copy link
Member

adleong commented Dec 20, 2016

Perhaps moving the StatsFilter up and out of the endpoints stack and into the client stack above the StatsFactoryWrapper?

Tim-Brooks pushed a commit to Tim-Brooks/linkerd that referenced this issue Dec 20, 2018
@hawkw
proxy: Add histogram unit tests (linkerd#870)
9156a80 
This PR adds the unit tests for the proxy metrics module's Histogram 
implementation that I wrote in linkerd#775 to @olix0r's Histogram implementation
added in linkerd#868. The tests weren't too difficult to adapt for the new code,
and everything seems to work correctly!

Signed-off-by: Eliza Weisman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wmorgan @adleong