-
Notifications
You must be signed in to change notification settings - Fork 541
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Combine Quota functionality into the Check method. #124
Conversation
mixer/v1/check.proto
Outdated
// | ||
// This is primarily meaningful for quotas and indicates the number of quota tokens | ||
// desired. For non-quota conditions, this can be set to 0. | ||
int64 amount = 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would desired_use_count
be a better name here (to parallel the response of use_count
)? or maybe just uses
? amount
feels very quota-centric.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we need:
bool best_effort = 3;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have use cases where best_effort=false applies?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
best_effort
makes sense for quota, but not check. I already didn't love amount
here since it's only used for half the call. If we want that extra info, why not:
message Condition {
string name = 1;
message QuotaParams {
int64 amount = 1;
bool best_effort = 2;
}
message CheckParams {
// none are required today
}
oneof operation {
QuotaParams quota = 2;
CheckParams check = 3;
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
quota-prefetch will need it.
if prefetch is not enabled, best_effort = false. if prefetch is enabled, best_effort = true. users may or may not enable quota-prefetch.
mixer/v1/check.proto
Outdated
// | ||
// - `precondition` to indicate a generic precondition check. | ||
// - `quota:<quota name>` to indicate a quota check. | ||
string name = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since this isn't a map of <name, condition> or similar, what is the expected behavior for multiple Conditions
with the name
of "precondition" with differing amounts? Does mixer take the first? the last? the minimum?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made it a map, that way I don't need to answer you :-)
// The preconditions enforced depend on the set of supplied attributes and | ||
// the active configuration. | ||
rpc Check(CheckRequest) returns (CheckResponse) {} | ||
|
||
// Quota allocates and releases quota. | ||
rpc Quota(QuotaRequest) returns (QuotaResponse) {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for my understanding: are we getting rid of Quota altogether, or just for now? is the idea that resource quotas would use the unified Check() too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think that even resource quotas would eventually go through Check.
For the foreseeable future, I expect that the intermediated API will only support rate quotas, and resource quotas will only be exposed through the disintermediated API.
mixer/v1/check.proto
Outdated
string name = 1; | ||
|
||
// The amount of time for which this result can be considered valid. | ||
google.protobuf.Duration duration = 2 [(gogoproto.nullable) = false, (gogoproto.stdduration) = true]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe prepend valid_
here, since we no longer have the context of CachabilityInfo? so, valid_duration
and valid_use_count
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I understand correctly, this is an attempt to batch Check and Quota calls into one call.
If client doesn't have cache, this will save client one remote-call. But in the cost of making API looks complicated.
If client has cache for both Check and Quota, this batch will not save. The first call may be able to use it, with different expiration and granted_count, Check and Quota will make separate calls afterward.
But the API really looks complicated. Do we want to do that? Can we assume client always has cache? At least mixer client is going to have cache.
mixer/v1/check.proto
Outdated
// and negative responses (when `CheckResponse.status.Code is not OK) can be cached. | ||
CachabilityInfo cachability = 3 [(gogoproto.nullable) = false]; | ||
// Provides information regarding each individual tested condition. | ||
repeated ConditionResult results = 3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think status should be inside ConditionResult too? Each condition may have different status result. Server should not combine them, or client cache may not be able to use it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My intent was to not support partial failures. If a precondition check fails, then we shouldn't allocate any quotas.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about precondition passed, but quota fails, how can client tell a failed status is from which condition? For just Check and Quota, if Quota failure has a fixed status code, we may be able to work around this problem. But semantic wise, a repeated conditions may means a lot of conditions to check, client like to know the result of each condition.
Updated per Doug's comments. Wayne, you make a good point that an efficient cache will inherently reduce the RPC overhead of Check+Quota. However, the overhead will not be negligible and this new API would cut this overhead by almost 50%. Additionally, it's worth considering the behavior of the system when check and quota results aren't in the cache. In order to maintain correct quota accounting, the client will need to issue Check, wait for Check to return, and then issue Quota. This means that the latency for uncached accesses is in fact 2x the cost of a single API. Ouch. Combining these two operations together avoids this double wait. Do you really feel the API is that complicated? CheckRequest and CheckResponse remain fairly small structure with narrow semantics. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is very difficult for mixer client to use this API if mixer client has check cache and quota prefetch.
They will have different cache expiration. it is very hard for one to wait for another so their remote calls can be batched into one call.
For easy implementation, I will just make two calls.
mixer/v1/check.proto
Outdated
// | ||
// This is primarily meaningful for quotas and indicates the number of quota tokens | ||
// desired. For non-quota conditions, this can be set to 0. | ||
int64 amount = 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we need:
bool best_effort = 3;
@qiwzhang In terms of usability, I think we need to discuss this in person. Previously you had to do:
Now, you supply all the same information in a single CheckRequest and you get the same things back in response. It's actually simpler to use since there is no synchronization between two API calls. |
Your steps are without cache. With Check and Quota cache, the steps are:
check_transport_cb is making the actual remote call. it is cache to determine when to call it since they have different expiration for check and quota. |
// To detect global dictionary out of sync between client and server. | ||
// The number of words in the global dictionary, used with to populate the attributes. | ||
// This value is used as a quick way to determine whether the client is using a dictionary that | ||
// the server understands. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did we decide to use number of words in the dictionary as the differentiator rather than something even marginally stronger like a checksum?
As I've said in other places, I think we need to plan for better handling of the dictionary. Why not mandate a version number for the dictionary and use that here instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To close the loop, based on out of band conversations: the global dictionary is append only and does not allow deletions. Therefore number of words allows both client and server to fall back on a dictionary they agree on, by using only min(len(server.global_dict), len(client.global_dict))
.
mixer/v1/check.proto
Outdated
// | ||
// This is primarily meaningful for quotas and indicates the number of quota tokens | ||
// desired. For non-quota conditions, this can be set to 0. | ||
int64 amount = 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
best_effort
makes sense for quota, but not check. I already didn't love amount
here since it's only used for half the call. If we want that extra info, why not:
message Condition {
string name = 1;
message QuotaParams {
int64 amount = 1;
bool best_effort = 2;
}
message CheckParams {
// none are required today
}
oneof operation {
QuotaParams quota = 2;
CheckParams check = 3;
}
}
PTAL. This uses a more general-purpose approach for conditions. Maps are gone, and instead simpler arrays are used instead. The best_effort functionality is back for quota. This continues to have a single deduplication_id per request, as opposed to per condition. Similarly, this continues to have a single status result per response, instead of one per condition. I don't believe we can reasonably support partial failures of this API. As it is defined now, it's all or nothing, which is easier for consumers to deal with. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
// To detect global dictionary out of sync between client and server. | ||
// The number of words in the global dictionary, used with to populate the attributes. | ||
// This value is used as a quick way to determine whether the client is using a dictionary that | ||
// the server understands. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To close the loop, based on out of band conversations: the global dictionary is append only and does not allow deletions. Therefore number of words allows both client and server to fall back on a dictionary they agree on, by using only min(len(server.global_dict), len(client.global_dict))
.
mixer/v1/check.proto
Outdated
oneof param { | ||
PreconditionParams precondition = 3; | ||
QuotaParams quota = 2; | ||
} | ||
} | ||
|
||
message CheckResponse { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I understand it, the long term goal is for Mixer to return the set of attributes it used to determine the result, and for Envoy to cache on those attributes. Should we include an Attributes
value on the CheckResponse
to use for that? Right now the comments on ConditionResult
seem to indicate that all attributes sent to Mixer will be used to construct the cache key, which will result cache misses that could've been hits.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we'll need something like that. I'm deferring that design to a later PR.
mixer/v1/check.proto
Outdated
CachabilityInfo cachability = 3 [(gogoproto.nullable) = false]; | ||
// Provides information regarding each individual tested condition. | ||
// There's one result per tested condition as specified in `CheckRequest.conditions` | ||
repeated ConditionResult results = 3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CheckRequest.conditions[i]
corresponds to CheckResponse.results[i]
, given that the results have no name? We should make note of that in the comments (you allude to it but don't come out and say explicitly).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
count() and expiration() also applied to failed condition. Even a failed condition can be cached.
Is that correct?
mixer/v1/check.proto
Outdated
|
||
// Used for deduplicating `Check` calls in the case of failed RPCs and retries. This should be a UUID | ||
// per call, where the same UUID is used for retries of the same call. | ||
string deduplication_id = 4; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the confusion is: do I need to set it if quota is not used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it needs to be set in all uses of the API. I'll probably add it to Report as well. This is just a way to ensure that mutating operations only occur once. It matters for quota, and it matters for thing like Report for billing events.
So whenever you send a message, just set the dedup id. If the RPC fails, you can retry the same message with the same dedup ID.
// A status code of OK indicates all preconditions were satisfied. Any other code indicates not | ||
// all preconditions were satisfied. | ||
// A status code of OK indicates all conditions were satisfied. Any other code indicates not | ||
// all conditions were satisfied. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if client has separate check cache and quota cache, it needs to know an error status from which condition? How can the client know which condition results in the error?
mixer/v1/check.proto
Outdated
CachabilityInfo cachability = 3 [(gogoproto.nullable) = false]; | ||
// Provides information regarding each individual tested condition. | ||
// There's one result per tested condition as specified in `CheckRequest.conditions` | ||
repeated ConditionResult results = 3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
count() and expiration() also applied to failed condition. Even a failed condition can be cached.
Is that correct?
OK, this is yet another variation of the API surface. It's simpler overall than before and supports the semantics Wayne needs for proper caching. PTAL. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: typical pattern is to declare the new messages then use them, rather than using them then declaring all the messages at the end of the parent message.
I do like that this better represents the shape of the return (a single precondition, N quotas).
mixer/v1/check.proto
Outdated
// The number of uses for which a `Check` response can be considered valid. | ||
int32 use_count = 4; | ||
// The number of uses for which this result can be considered valid. | ||
int32 valid_use_count = 3; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be granted_amount to match "amount" in the request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it should be 0 if best_effort = false, and amount is not enough. it can be less than "amount" if best_effort is true.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
type is int64 too to match "amount" type in the request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
PTAL. |
Check is now expressed as a series of conditions to check. It can check preconditions (what we've traditionally associated with Check), and it can check/allocate quotas.
* Point to googleapi in service control client. (istio#91) * Point to googleapi in service control client. * Use git repository for service-control-client. * Merge latest changes from master (istio#104) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Extract quota config from service config. (istio#101) * Add metric_cost in config. * Remove group rules. * Call loadQuotaConfig in config::create. * Update latest update from master branch (istio#106) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Added quota contoll without the service control client library (istio#93) * Added quota contoll without the service control client library * Applied code review * Applied code review * Resolve conflicts * Resolve conflicts * Fixed format error reported by script/check-style * Fixed a bug at Aggregated::GetAuthToken that causes Segmentation Fault * Changed usage of template funcion * Applied latest changes from the repo * Applied latest changes from the repo * Applied latest changes from the repo * Adde comments * Updated log information * Applied istio#101 * Changed metric_cost_map to metric_cost_vector * Fixed test case compilation error * Fixed test case compilation error * Add unit test for quota config. (istio#108) * Add unit test for quota config. * Add comments. * Update test specifics. * Merge latest changes from master branch (istio#112) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Not to use api_key if its service is not actived. (istio#109) * If QuotaControl service is not available, return utils::Status::OK (istio#113) * If QuotaControl service is not available, return utils::Status::OK * Updated comment * Return HTTP status code 429 on google.rpc.Code.RESOURCE_EXHAUSTED (istio#119) * Fixed incorrectly resolved conflicts (istio#123) * Added unit test cases for rate limiting (istio#124) * Fixed incorrectly resolved conflicts * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Rename response.http.code (istio#125) (istio#128) * Added handling of error code QUOTA_SYSTEM_UNAVAILABLE (istio#148) * Integrated service control client library with quota cache aggregation (istio#149) * Fixed error on merge (istio#151) * Integrated service control client library with quota cache aggregation * Fixed error on merge * Fixed the compatibility issue with the latest update on esp (istio#152) * Removed copied proto files (istio#208) * Set default allocate quota request timeout to 1sec and applied latest service control client library change (istio#211) * Merged key_restriction related changes from master (istio#213) * Merge latest changes from master branch (istio#217) * Not call report if decodeHeaders is not called. (istio#150) * Update mixerclient with sync-ed grpc write and fail-fast. (istio#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (istio#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Uses a specific version of the Shared Pipeline lib (istio#158) * Update lyft/envoy commit Id to latest. (istio#161) * Update lyft/envoy commit Id to latest. * Remove the comment about pull request * Add new line - will delete in next commit. * Update repositories.bzl (istio#169) * Always set response latency (istio#172) * Update mixerclient to sync_transport change. (istio#178) * Use opaque config to turn on/off forward attribute and mixer filter (istio#179) * Modify mixer filter * Swap defaults * Make the filter decoder only * cache mixer disabled decision * Fix a bug in opaque config change and test it out (istio#182) * Fix a bug and test it out * Update filter type * Update README.md * Update mixer client to mixer api with gogoproto. (istio#184) * Move .bazelrc to tools/bazel.rc (istio#186) * Move .bazelrc to tools/bazel.rc * Update Jenkinsfile with latest version of pipeline * Support apikey based traffic restriction (istio#189) * b/36368559 support apikey based traffic restriction * Fixed code formatting * Fix crash in unreachable/overloaded RDS (istio#190) * Add mixer client end to end integration test. (istio#177) * Add mixer client end to end integration test. * Split some repositories into a separate file. * use real mixer for fake mixer_server. * Test repository * use mixer bzl file. * Use mixer repositories * Not to use mixer repository. * Add return line at the end of WORKSPACE. * Fix broken link (istio#193) * Make quota call (istio#192) * hookup quota call * Make quota call. * Update indent. * Update envoy and update configs (istio#195) * Update envoy and update configs * Use gcc-4.9 for travis * Use bazel 0.4.5 * Fix SHA of lightstep-tracer-common * Enable check cache and refactory mixer config loading (istio#197) * Refactory the mixer config loading. * fix format * Add integration test. * updated README.md * s/send/sent/ * Split into separate tests. (istio#201) * Update README on how to enable check cache. (istio#204) * Update README on how to enable check cache. * Update the comment. * build: support Envoy native Bazel build. (istio#210) * build: support Envoy native Bazel build. This patch switches the Envoy build from src/envoy/repositories.bzl to using the upstream native build. See envoyproxy/envoy#663 for the corresponding changes on the Envoy side. * Use Envoy master with BUILD.wip rename merged. * Fix clang-format issues. * Fixes bazel.rc issues (istio#212) * Fixes bazel rc issues * Update Jenkins to latest pipeline version * Updated the commit id of cloudendpoints/service-control-client-cxx (istio#218) * Update commitid of cloudendpoints/service-control-client-cxx repo (istio#220)
* Created check security rules file and a few dummy/helper functions. (istio#40) * Created check security rules file and a few dummy/helper functions. And added it to check work flow. * Fix format. * Firebase: Merge from master. (istio#53) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (istio#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (istio#41) * Update prototype to use iptables (istio#42) * Rebase to fixed Envoy (istio#43) * Handle HEAD request. (istio#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (istio#48) * Try again (istio#49) * Enable ESP to invoke Firebase Security rules. (istio#54) * Enable ESP to invoke Firebase Security rules. * Address code review comments. * Remove some debug logs * Add proto file to capture TestRulesetRequest. * clang-format files * Resolve a merge issue with previous commit * Allow security rules to disabled via serverconfig * format file * Addressed Wayne's review comments. * Add firebase server to Server Config. * Address Lizan's review comments * Address review comments. * Disable check rules service by default. * Address more review comments. * Fix a check. * Delete unwanted constant. * Address Wayne's comments and add a simple config test. * Address a review comment. * Add negative test case for config * Address code review * Remove unwanted const std::string * Merge from master into firebase (istio#65) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (istio#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (istio#41) * Update prototype to use iptables (istio#42) * Rebase to fixed Envoy (istio#43) * Handle HEAD request. (istio#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (istio#48) * Try again (istio#49) * Integrate with mixer client. (istio#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (istio#56) * Add uuid-dev dependency in README.md (istio#45) * Extract originIp and OriginHost. (istio#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (istio#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (istio#59) * Use envoy new access_log handler for sending Report. (istio#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with istio#396. (istio#61) * Fix tclap dependency fetching error (istio#62) * Update the auth checke to use service.experimental.authorization.providerwq! * Update the auth check to use service.experimental.authorization.provider * Update the auth check to use service.experimental.authorization.provider (istio#67) * Update the auth check to use service.experimental.authorization.provider * Address comments and revert accidental change. * Remove unnecessary added accidentally. * Another patch * fix the logic * fix lint * Fix broken test and add unit tests * Fix comments * Fix style check * revert style for raw string * fix small lint * fix small lint * fix small lint * Unit tests for check security rules. (istio#75) * Unit tests for check security rules. * format * Address review comments. * Fix typos * Merge from master to firebase (istio#143) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (istio#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (istio#41) * Update prototype to use iptables (istio#42) * Rebase to fixed Envoy (istio#43) * Handle HEAD request. (istio#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (istio#48) * Try again (istio#49) * Integrate with mixer client. (istio#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (istio#56) * Add uuid-dev dependency in README.md (istio#45) * Extract originIp and OriginHost. (istio#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (istio#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (istio#59) * Use envoy new access_log handler for sending Report. (istio#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with istio#396. (istio#61) * Fix tclap dependency fetching error (istio#62) * Integrate mixer client directly with envoy. (istio#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (istio#68) * Push tar.gz to GCS (istio#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (istio#72) * Update mixer client SHA. (istio#74) * Update readme. (istio#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (istio#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (istio#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (istio#78) * Add script to build docker image. (istio#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (istio#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (istio#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (istio#82) * Fix src/envoy/mixer/README.md (istio#85) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Not to use api_key if its service is not actived. (istio#109) * Update envoy and add c-ares (istio#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (istio#110) * Add send_attribute filter. (istio#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (istio#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (istio#125) * Send headers as string map. (istio#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (istio#127) * update base debug docker image reference (istio#133) * Update postsubmit to create docker images (istio#132) * Adding config release for bazel build (istio#135) * Fix mixer client crash. (istio#136) * Get mixerclient with response parsing. (istio#138) * Update nghttp2 to sync with envoy (istio#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Merge from master to firebase (istio#159) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (istio#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (istio#41) * Update prototype to use iptables (istio#42) * Rebase to fixed Envoy (istio#43) * Handle HEAD request. (istio#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (istio#48) * Try again (istio#49) * Integrate with mixer client. (istio#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (istio#56) * Add uuid-dev dependency in README.md (istio#45) * Extract originIp and OriginHost. (istio#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (istio#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (istio#59) * Use envoy new access_log handler for sending Report. (istio#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with istio#396. (istio#61) * Fix tclap dependency fetching error (istio#62) * Integrate mixer client directly with envoy. (istio#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (istio#68) * Push tar.gz to GCS (istio#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (istio#72) * Update mixer client SHA. (istio#74) * Update readme. (istio#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (istio#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (istio#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (istio#78) * Add script to build docker image. (istio#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (istio#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (istio#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (istio#82) * Fix src/envoy/mixer/README.md (istio#85) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Not to use api_key if its service is not actived. (istio#109) * Update envoy and add c-ares (istio#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (istio#110) * Add send_attribute filter. (istio#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (istio#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (istio#125) * Send headers as string map. (istio#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (istio#127) * update base debug docker image reference (istio#133) * Update postsubmit to create docker images (istio#132) * Adding config release for bazel build (istio#135) * Fix mixer client crash. (istio#136) * Get mixerclient with response parsing. (istio#138) * Update nghttp2 to sync with envoy (istio#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Populate origin.user attribute from the SAN field of client cert (istio#142) * Test * test * test * revert file * address comments * test * fix typo * fix format * fix format * Update to latest mixer_client. (istio#145) * Update to latest mixer_client. * Updated the sha. * Not call report if decodeHeaders is not called. (istio#150) * Update mixerclient with sync-ed grpc write and fail-fast. (istio#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (istio#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Update the Commit id for envoy * Allow for HTTP based function from Firebase rules (istio#202) * Allow for HTTP based function from Firebase rules * Fix code style check * Added more comments. * Fix style issues. * Address code review comments from Limin and Lizan. * Add more comments and address CR comments. * Fix a typo. * Address Wayne's CR comments. * Merge from master to firebase (istio#237) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (istio#38) * Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect * Fixed style. * Rebase Envoy (istio#41) * Update prototype to use iptables (istio#42) * Rebase to fixed Envoy (istio#43) * Handle HEAD request. (istio#34) * Handle HEAD request. * Try with GET if HEAD fails. * Address comments. * Format file. * Expose bazel target (istio#48) * Try again (istio#49) * Integrate with mixer client. (istio#55) * Integrate with mixer client. * Restore repositories.bzl back. * Add originIp and originHost attributes. (istio#56) * Add uuid-dev dependency in README.md (istio#45) * Extract originIp and OriginHost. (istio#57) * Extract originIp and OriginHost. * Make header x-forwarded-host const. * Update buckets for UI. (istio#58) * Update buckets for UI. * Only update time_distribution. * Add targetService attribute. (istio#59) * Use envoy new access_log handler for sending Report. (istio#60) * use access_log handler. * Not to use Loggable base class. * Update to the latest envoy with istio#396. (istio#61) * Fix tclap dependency fetching error (istio#62) * Integrate mixer client directly with envoy. (istio#66) * Integrate mixer client directly with envoy. * Send response header in Report. * rename filter name from esp to mixer. * add README. * Add release binary script. (istio#68) * Push tar.gz to GCS (istio#69) * Push tar.gz to GCS * Rename envoy_esp * Remove mixer_client from api_manager. (istio#72) * Update mixer client SHA. (istio#74) * Update readme. (istio#73) * Adds Jenkinsfile and updates release-binary to create a SHA. (istio#71) * Adds Jenkinsfile and update release-binary * Update Jenkinsfile and gitignore * Fixes typo and use normal build Node * Uses default bazel config * Using batch mode * Update bazel memory settings * Do not use Jenkins bazel env * Set .bazelrc for postsubmit * Update grpc and protobuf (istio#70) * protobuf v3.2.0 * grpc v1.1.1 * Align auth lib with grpc 1.1.1 * Add sourceService. (istio#78) * Add script to build docker image. (istio#77) * Add script to build docker image. * Add start_envoy for docker image. * Use official attribute names (istio#80) * Use official attribute names * fix format * Creates a KEY for mixer client dep. Updates release-binary (istio#79) * Updated mixer repo to use a key for commit * release-binary skip build if file exists. * Update src/envoy/mixer/README. (istio#82) * Fix src/envoy/mixer/README.md (istio#85) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Not to use api_key if its service is not actived. (istio#109) * Update envoy and add c-ares (istio#107) * Update envoy and add c-ares depedencies * Update release script with debug and normal binary * remove debug ls * formatting * Send StatusCode Attributes to Mixer. (istio#110) * Add send_attribute filter. (istio#115) * Add send_attribute filter. * Fix format * rename variable serialized_attributes_ * Address the comments. * Fail request if api_key is not valid. (istio#116) * Fail request if api_key is not valid. * Format code. * Update comments. * Address comment. * Rename response.http.code (istio#125) * Send headers as string map. (istio#129) * Send headers as string map. * Remove origin.ip and origin.host. * Fix format * unify bazel's docker build targets with other istio repos (istio#127) * update base debug docker image reference (istio#133) * Update postsubmit to create docker images (istio#132) * Adding config release for bazel build (istio#135) * Fix mixer client crash. (istio#136) * Get mixerclient with response parsing. (istio#138) * Update nghttp2 to sync with envoy (istio#140) * Fix src/envoy/mixer/README.md * Update nghttp2 to sync with envoy * update * fix typo * Populate origin.user attribute from the SAN field of client cert (istio#142) * Test * test * test * revert file * address comments * test * fix typo * fix format * fix format * Update to latest mixer_client. (istio#145) * Update to latest mixer_client. * Updated the sha. * Not call report if decodeHeaders is not called. (istio#150) * Update mixerclient with sync-ed grpc write and fail-fast. (istio#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (istio#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Uses a specific version of the Shared Pipeline lib (istio#158) * Update lyft/envoy commit Id to latest. (istio#161) * Update lyft/envoy commit Id to latest. * Remove the comment about pull request * Add new line - will delete in next commit. * Update repositories.bzl (istio#169) * Always set response latency (istio#172) * Update mixerclient to sync_transport change. (istio#178) * Use opaque config to turn on/off forward attribute and mixer filter (istio#179) * Modify mixer filter * Swap defaults * Make the filter decoder only * cache mixer disabled decision * Fix a bug in opaque config change and test it out (istio#182) * Fix a bug and test it out * Update filter type * Update README.md * Update mixer client to mixer api with gogoproto. (istio#184) * Move .bazelrc to tools/bazel.rc (istio#186) * Move .bazelrc to tools/bazel.rc * Update Jenkinsfile with latest version of pipeline * Support apikey based traffic restriction (istio#189) * b/36368559 support apikey based traffic restriction * Fixed code formatting * Fix crash in unreachable/overloaded RDS (istio#190) * Add mixer client end to end integration test. (istio#177) * Add mixer client end to end integration test. * Split some repositories into a separate file. * use real mixer for fake mixer_server. * Test repository * use mixer bzl file. * Use mixer repositories * Not to use mixer repository. * Add return line at the end of WORKSPACE. * Fix broken link (istio#193) * Make quota call (istio#192) * hookup quota call * Make quota call. * Update indent. * Update envoy and update configs (istio#195) * Update envoy and update configs * Use gcc-4.9 for travis * Use bazel 0.4.5 * Fix SHA of lightstep-tracer-common * Enable check cache and refactory mixer config loading (istio#197) * Refactory the mixer config loading. * fix format * Add integration test. * updated README.md * s/send/sent/ * Split into separate tests. (istio#201) * Update README on how to enable check cache. (istio#204) * Update README on how to enable check cache. * Update the comment. * build: support Envoy native Bazel build. (istio#210) * build: support Envoy native Bazel build. This patch switches the Envoy build from src/envoy/repositories.bzl to using the upstream native build. See envoyproxy/envoy#663 for the corresponding changes on the Envoy side. * Use Envoy master with BUILD.wip rename merged. * Fix clang-format issues. * Fixes bazel.rc issues (istio#212) * Fixes bazel rc issues * Update Jenkins to latest pipeline version * Fix go build (istio#224) * Use TranscoderInputStream to reduce confusion around ByteCount() (istio#225) * Add TranscoderInputStream to reduce confusion * fix_format * Merge latest changes from rate_limiting to master (istio#221) * Point to googleapi in service control client. (istio#91) * Point to googleapi in service control client. * Use git repository for service-control-client. * Merge latest changes from master (istio#104) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Extract quota config from service config. (istio#101) * Add metric_cost in config. * Remove group rules. * Call loadQuotaConfig in config::create. * Update latest update from master branch (istio#106) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Added quota contoll without the service control client library (istio#93) * Added quota contoll without the service control client library * Applied code review * Applied code review * Resolve conflicts * Resolve conflicts * Fixed format error reported by script/check-style * Fixed a bug at Aggregated::GetAuthToken that causes Segmentation Fault * Changed usage of template funcion * Applied latest changes from the repo * Applied latest changes from the repo * Applied latest changes from the repo * Adde comments * Updated log information * Applied istio#101 * Changed metric_cost_map to metric_cost_vector * Fixed test case compilation error * Fixed test case compilation error * Add unit test for quota config. (istio#108) * Add unit test for quota config. * Add comments. * Update test specifics. * Merge latest changes from master branch (istio#112) * Get attributes from envoy config. (istio#87) * Send all attributes. * Remove unused const strings. * Address comment. * updated SHA to point to newer envoy with RDS API feature (istio#94) * Disable travis on stable branches (istio#96) * Publish debug binaries (no release yet) (istio#98) * Copies the binary instead of linking for release (istio#102) * Not to use api_key if its service is not actived. (istio#109) * If QuotaControl service is not available, return utils::Status::OK (istio#113) * If QuotaControl service is not available, return utils::Status::OK * Updated comment * Return HTTP status code 429 on google.rpc.Code.RESOURCE_EXHAUSTED (istio#119) * Fixed incorrectly resolved conflicts (istio#123) * Added unit test cases for rate limiting (istio#124) * Fixed incorrectly resolved conflicts * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Added unit test cases for rate limiting * Rename response.http.code (istio#125) (istio#128) * Added handling of error code QUOTA_SYSTEM_UNAVAILABLE (istio#148) * Integrated service control client library with quota cache aggregation (istio#149) * Fixed error on merge (istio#151) * Integrated service control client library with quota cache aggregation * Fixed error on merge * Fixed the compatibility issue with the latest update on esp (istio#152) * Removed copied proto files (istio#208) * Set default allocate quota request timeout to 1sec and applied latest service control client library change (istio#211) * Merged key_restriction related changes from master (istio#213) * Merge latest changes from master branch (istio#217) * Not call report if decodeHeaders is not called. (istio#150) * Update mixerclient with sync-ed grpc write and fail-fast. (istio#155) * Update mixerclient with sync-ed write and fail-fast. * Update to latest test. * Update again * Update envoy to PR553 (istio#156) * Update envoy to PR553 * Update libevent to 2.1.8 * Uses a specific version of the Shared Pipeline lib (istio#158) * Update lyft/envoy commit Id to latest. (istio#161) * Update lyft/envoy commit Id to latest. * Remove the comment about pull request * Add new line - will delete in next commit. * Update repositories.bzl (istio#169) * Always set response latency (istio#172) * Update mixerclient to sync_transport change. (istio#178) * Use opaque config to turn on/off forward attribute and mixer filter (istio#179) * Modify mixer filter * Swap defaults * Make the filter decoder only * cache mixer disabled decision * Fix a bug in opaque config change and test it out (istio#182) * Fix a bug and test it out * Update filter type * Update README.md * Update mixer client to mixer api with gogoproto. (istio#184) * Move .bazelrc to tools/bazel.rc (istio#186) * Move .bazelrc to tools/bazel.rc * Update Jenkinsfile with latest version of pipeline * Support apikey based traffic restriction (istio#189) * b/36368559 support apikey based traffic restriction * Fixed code formatting * Fix crash in unreachable/overloaded RDS (istio#190) * Add mixer client end to end integration test. (istio#177) * Add mixer client end to end integration test. * Split some repositories into a separate file. * use real mixer for fake mixer_server. * Test repository * use mixer bzl file. * Use mixer repositories * Not to use mixer repository. * Add return line at the end of WORKSPACE. * Fix broken link (istio#193) * Make quota call (istio#192) * hookup quota call * Make quota call. * Update indent. * Update envoy and update configs (istio#195) * Update envoy and update configs * Use gcc-4.9 for travis * Use bazel 0.4.5 * Fix SHA of lightstep-tracer-common * Enable check cache and refactory mixer config loading (istio#197) * Refactory the mixer config loading. * fix format * Add integration test. * updated README.md * s/send/sent/ * Split into separate tests. (istio#201) * Update README on how to enable check cache. (istio#204) * Update README on how to enable check cache. * Update the comment. * build: support Envoy native Bazel build. (istio#210) * build: support Envoy native Bazel build. This patch switches the Envoy build from src/envoy/repositories.bzl to using the upstream native build. See envoyproxy/envoy#663 for the corresponding changes on the Envoy side. * Use Envoy master with BUILD.wip rename merged. * Fix clang-format issues. * Fixes bazel.rc issues (istio#212) * Fixes bazel rc issues * Update Jenkins to latest pipeline version * Updated the commit id of cloudendpoints/service-control-client-cxx (istio#218) * Update commitid of cloudendpoints/service-control-client-cxx repo (istio#220) * Send delta metrics for intermediate reports. (istio#219) * Send delta metrics for intermediate reports. * Move last_request_bytes/last_response_bytes to RequestContext. * Handle final report. * Address comment. * Update attributes to match the canonical attribute list. (istio#232) * Update response.http.code to response.code and response.latency to response.duration to line up with the canonical attributes in istio/istio.github.io/docs/concepts/attributes.md * Format according to clang-format * Add envoy Buffer based TranscoderInputStream (istio#231) * Add envoy Buffer based TranscoderInputStream * fix format * A few doc changes for consistency across repos. (istio#235) * Add repositories.bzl * Added missing export setting in bazel configuration (istio#236) * Added export missing in bazel configuration * Added export missing in bazel configuration * Allow HTTP functions in firebase rules to specify audience (istio#244) * Allow HTTP functions in firebase rules to specify audience * Allow GetAuthToken to ignore cache and fix style checks. * Fix GetAuthToken * Address Wayne's comment * Check for empty response body * Remove .bazelrc.jenkins file not present in the master branch. * Remove forward_attribute_filter.cc not present in master.
Signed-off-by: Liam White <[email protected]>
Check is now expressed as a series of conditions to check.
It can check preconditions (what we've traditionally associated
with Check), and it can check/allocate quotas.
This approach lets us reduce the RPC overhead from 2 calls to 1 and lets Mixer
execute precondition checks and quota allocations in parallel.
THIS WILL BE A BREAKING API CHANGE.