diff --git a/doc/reference/controllers.md b/doc/reference/controllers.md index d6091e958c..d9bbbd087a 100644 --- a/doc/reference/controllers.md +++ b/doc/reference/controllers.md @@ -9,10 +9,10 @@ - [Pipeline](#pipeline) - [StatusSyncController](#statussynccontroller) - [Business Controllers](#business-controllers) + - [GlobalFilter](#globalfilter) - [EaseMonitorMetrics](#easemonitormetrics) - [FaaSController](#faascontroller) - [IngressController](#ingresscontroller) - - [MeshController](#meshcontroller) - [ConsulServiceRegistry](#consulserviceregistry) - [EtcdServiceRegistry](#etcdserviceregistry) - [EurekaServiceRegistry](#eurekaserviceregistry) @@ -26,12 +26,15 @@ - [httpserver.Rule](#httpserverrule) - [httpserver.Path](#httpserverpath) - [httpserver.Header](#httpserverheader) - - [pipeline.FlowNode](#pipeline.FlowNode) - - [filters.Filter](#filtersFilter) + - [pipeline.Spec](#pipelinespec) + - [pipeline.FlowNode](#pipelineflownode) + - [filters.Filter](#filtersfilter) - [easemonitormetrics.Kafka](#easemonitormetricskafka) - [nacos.ServerSpec](#nacosserverspec) - [autocertmanager.DomainSpec](#autocertmanagerdomainspec) - - [resilience.Policy](#resiliencePolicy) + - [resilience.Policy](#resiliencepolicy) + - [Retry Policy](#retry-policy) + - [Circuit Break Policy](#circuit-break-policy) As the [architecture diagram](../imgs/architecture.png) shows, the controller is the core entity to control kinds of working. There are two kinds of controllers overall: @@ -101,7 +104,7 @@ rules: | ipFilter | [ipfilter.Spec](#ipfilterSpec) | IP Filter for all traffic under the server | No | | rules | [httpserver.Rule](#httpserverRule) | Router rules | No | | autoCert | bool | Do HTTP certification automatically | No | -| clientMaxBodySize | int64 | Max size of request body. the default value is 4MB. Requests with a body larger than this option are discarded. When this option is set to `-1`, Easegress takes the request body as a stream and the body can be any size, but some features are not possible in this case. | No | +| clientMaxBodySize | int64 | Max size of request body. the default value is 4MB. Requests with a body larger than this option are discarded. When this option is set to `-1`, Easegress takes the request body as a stream and the body can be any size, but some features are not possible in this case, please refer [Stream](./stream.md) for more information. | No | | caCertBase64 | string | Define the root certificate authorities that servers use if required to verify a client certificate by the policy in TLS Client Authentication. | No | | globalFilter | string | Name of [GlobalFilter](#globalfilter) for all backends | No | @@ -229,13 +232,6 @@ filters: ``` In this case, we give second `proxy` alias `proxy2`, so request is invalid, it jumps to second proxy. -#### pipeline.Spec -| Name | Type | Description | Required | -|------|------|-------------|----------| -| flow | [pipeline.FlowNode](#pipelineFlowNode) | Flow of pipeline | No | -| filters | [][filters.Filter](#filters.Filter) | Filter definitions of pipeline | Yes | -| resilience | [][resilience.Policy](#resiliencePolicy) | Resilience policy for backend filters | No | - ### StatusSyncController No config. @@ -502,7 +498,7 @@ domains: | methods | []string | Methods to match, empty means to allow all methods | No | | headers | [][httpserver.Header](#httpserverHeader) | Headers to match (the requests matching headers won't be put into cache) | No | | backend | string | backend name (pipeline name in static config, service name in mesh) | Yes | -| clientMaxBodySize | int64 | Max size of request body. the default value is 4MB. Requests with a body larger than this option are discarded. When this option is set to `-1`, Easegress takes the request body as a stream and the body can be any size, but some features are not possible in this case. | No | +| clientMaxBodySize | int64 | Max size of request body, will use the option of the HTTP server if not set. the default value is 4MB. Requests with a body larger than this option are discarded. When this option is set to `-1`, Easegress takes the request body as a stream and the body can be any size, but some features are not possible in this case, please refer [Stream](./stream.md) for more information. | No | | matchAllHeader | bool | Match all headers that are defined in headers, default is `false`. | No | @@ -516,6 +512,12 @@ There must be at least one of `values` and `regexp`. | values | []string | Header values to match | No | | regexp | string | Header value in regular expression to match | No | +### pipeline.Spec +| Name | Type | Description | Required | +|------|------|-------------|----------| +| flow | [pipeline.FlowNode](#pipelineFlowNode) | Flow of pipeline | No | +| filters | [][filters.Filter](#filters.Filter) | Filter definitions of pipeline | Yes | +| resilience | [][resilience.Policy](#resiliencePolicy) | Resilience policy for backend filters | No | ### pipeline.FlowNode diff --git a/doc/reference/filters.md b/doc/reference/filters.md index 906b6e622c..05921d8e8f 100644 --- a/doc/reference/filters.md +++ b/doc/reference/filters.md @@ -165,7 +165,7 @@ pools: | mtls | [proxy.MTLS](#proxymtls) | mTLS configuration | No | | maxIdleConns | int | Controls the maximum number of idle (keep-alive) connections across all hosts. Default is 10240 | No | | maxIdleConnsPerHost | int | Controls the maximum idle (keep-alive) connections to keep per-host. Default is 1024 | No | -| serverMaxBodySize | int64 | Max size of response body. the default value is 4MB. Responses with a body larger than this option are discarded. When this option is set to `-1`, Easegress takes the response body as a stream and the body can be any size, but some features are not possible in this case. | No | +| serverMaxBodySize | int64 | Max size of response body. the default value is 4MB. Responses with a body larger than this option are discarded. When this option is set to `-1`, Easegress takes the response body as a stream and the body can be any size, but some features are not possible in this case, please refer [Stream](./stream.md) for more information. | No | ### Results @@ -864,7 +864,7 @@ Rules to revise request header. | loadBalance | [proxy.LoadBalance](#proxyLoadBalanceSpec) | Load balance options | Yes | | memoryCache | [proxy.MemoryCacheSpec](#proxymemorycachespec) | Options for response caching | No | | filter | [proxy.RequestMatcherSpec](#proxyrequestmatcherspec) | Filter options for candidate pools | No | -| serverMaxBodySize | int64 | Request max body size | No | +| serverMaxBodySize | int64 | Max size of response body, will use the option of the Proxy if not set. Responses with a body larger than this option are discarded. When this option is set to `-1`, Easegress takes the response body as a stream and the body can be any size, but some features are not possible in this case, please refer [Stream](./stream.md) for more information. | No | | timeout | string | Request calceled when timeout | No | | retryPolicy | string | Retry policy name | No | | circuitBreakPolicy | string | Circuit break policy name | No | diff --git a/doc/reference/stream.md b/doc/reference/stream.md new file mode 100644 index 0000000000..6714dd0513 --- /dev/null +++ b/doc/reference/stream.md @@ -0,0 +1,35 @@ +# Stream + +Most Easegress traffic is message-based, but Easegress v2 pipeline is +protocol independent, that's it supports stream-based traffic like TCP, and +there's stream traffic even in HTTP. + +Another new feature in Easegress v2 is multiple requests/responses support, +this requires the payload of a request/response can be read more than once, +for message-based traffic, this is simple and easy to do, as Easegress can +read the full message payload into memory. But for stream-based traffic, +this is impossible, as the payload may require too much memory, and/or take +too much time to read it into memory. + +To resolve the above issue, Easegress allow user or developer to configure +whether a request/response is a stream, for example: + +* We can set `clientMaxBodySize` of an HTTP server to a negative value to + tell Easegress the request is a stream, and not a stream otherwise. Please + refer [HTTP Server](./controllers.md#httpserver) for more information. +* We can set `serverMaxBodySize` of a `Proxy` filter to a negative value to + tell Easegress the response is a stream, and not a stream otherwise. Please + refer [Proxy](./filters.md#proxy) for more information. + +As we have mentioned above, the payload of a stream-based request/response +cannot be read once, so some features are not possible for these +requests/responses, including: + +* In the `template` of `RequestBuilder` or `ResponseBuilder`, you cannot + access the payload(in HTTP, the body) of an existing stream-based + request/response, while it is fine to access other information of the + request/response. +* Stream-based request/response cannot be cached by `Proxy`. +* The `HeaderToJSON` filter does not support stream-based requests/responses. +* You cannot access the payload of stream-based request/response in a + `WasmHost` filter.