Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validation: Update a couple debug logs that can be large with large p… #7035

Merged
merged 11 commits into from
Aug 26, 2022

Conversation

nfuden
Copy link
Contributor

@nfuden nfuden commented Aug 25, 2022

Description

Updates calls to logger debug that now had proxy in the log response post merge.
This previously could cause issues in environments with large proxies that had debug enabled post gateway merge especially if they have several validation requests in process.

@github-actions github-actions bot added the keep pr updated signals bulldozer to keep pr up to date with base branch label Aug 25, 2022
reports, err := v.getReportsFromGlooValidationResponse(response)
// reports are gauranteed to not be nil at this point and we can more easily not report on the proxy
// than on the response itself without more loops on non-debug calls
logger.Debugf("Got response from GlooValidationService: UpstreamReports: %v, ProxyReports: %v", *reports.UpstreamReports, *reports.ProxyReports)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be too paranoid but given that we've also seen very large reports/statuses on proxies maybe we should only log the state and/or move all the proxy contents logging to "trace"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah so even proxyreports could be super large. Ill check it out

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, will this construct the (potentially large) formatted string regardless of whether we're logging at debug level? Is this a case where we'd we'd want to wrap in if debugEnabled (maybe go's logger is smarter than log4j, I'm not sure)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zap is smart enough to check that as the first statement prior to the execution checks the not exposed level. Given that I didnt want to make a loop gated by level as it wouldnt be pulling from the logger itself.

@github-actions
Copy link

github-actions bot commented Aug 25, 2022

Visit the preview URL for this PR (updated for commit c1c4c8b):

https://gloo-edge--pr7035-feat-vwh-deublogging-9kjl497l.web.app

(expires Fri, 02 Sep 2022 15:37:21 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

@solo-changelog-bot
Copy link

Issues linked to changelog:
#7031

Copy link
Contributor

@sam-heilbron sam-heilbron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each of the validator methods seem to have a debug log line where we log the entire response (ValidateDeleteSecret: logger.Debugf("Got response from GlooValidationService: %s", response.String())). Do we need to update these instances as well?

Side note: Given the ease of using the String method for the ValidationRequest and ValidationResponse, would it make sense to implementer some stringer method on those objects which can always be called and contain only the relevant details?

@nfuden
Copy link
Contributor Author

nfuden commented Aug 26, 2022

The problem is that it is the stringer and the appropriate small subobjects portion would have a decent loop overhead. We can address that at a later point.

@nfuden
Copy link
Contributor Author

nfuden commented Aug 26, 2022

/kick [Fail] Access Log in memory Access Logs File [It] can create string access logs

@soloio-bulldozer soloio-bulldozer bot merged commit 651340f into master Aug 26, 2022
@soloio-bulldozer soloio-bulldozer bot deleted the feat/vwh-deublogging branch August 26, 2022 16:26
nfuden added a commit that referenced this pull request Aug 26, 2022
#7035)

* validation: Update a couple debug logs that can be large with large proxies

* changelog: Add issue link

* validation: Log level changes to avoid more potentially large objects

* changelog

* validation: Last logs in validation service that contain potentially large objects removed

* Adding changelog file to new location

* Deleting changelog file from old location

Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: changelog-bot <changelog-bot>
gunnar-solo pushed a commit that referenced this pull request Sep 2, 2022
#7035)

* validation: Update a couple debug logs that can be large with large proxies

* changelog: Add issue link

* validation: Log level changes to avoid more potentially large objects

* changelog

* validation: Last logs in validation service that contain potentially large objects removed

* Adding changelog file to new location

* Deleting changelog file from old location

Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: changelog-bot <changelog-bot>
gunnar-solo pushed a commit that referenced this pull request Sep 2, 2022
#7035)

* validation: Update a couple debug logs that can be large with large proxies

* changelog: Add issue link

* validation: Log level changes to avoid more potentially large objects

* changelog

* validation: Last logs in validation service that contain potentially large objects removed

* Adding changelog file to new location

* Deleting changelog file from old location

Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: changelog-bot <changelog-bot>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
keep pr updated signals bulldozer to keep pr up to date with base branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants