-
Notifications
You must be signed in to change notification settings - Fork 7.7k
Experimental QUIC and HTTP 3 support in Istio gateways
Istio provides alpha support for HTTP/3 at the gateways. Currently, it is restricted to using mirror TLS-terminated HTTPS servers at the gateway. For instance, if there is a TLS-terminated HTTPS server at 443/TCP, then an HTTP/3 server at 443/UDP is created so that clients which support QUIC (like Google Chrome) can use HTTP/3.
Prerequisites for the demo
- Kubernetes cluster with
MixedProtocolLBService
feature gate enabled. - Set
PILOT_ENABLE_QUIC_LISTENERS
on the Pilot - Assuming 443/TCP is exposed, expose 443/UDP on the gateway and allow 443/UDP on the firewall
As of writing this, this feature was still in the main, unreleased branch. So the image is built from this branch. Once v1.12 is released then setting hub
and tag
won't be necessary to use this feature.
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
name: install
spec:
hub: localhost:5000
tag: master
components:
ingressGateways:
- name: istio-ingressgateway
enabled: true
k8s:
service:
ports:
- name: status-port
port: 15021
targetPort: 15021
# Both HTTPS and HTTP/3 must have
# the same port number. Here it
# is 443/TCP and 443/UDP
- name: https
port: 443
targetPort: 8443
- name: http3
port: 443
targetPort: 8443
protocol: UDP
values:
pilot:
# This is required to create mirror QUIC
# listeners for TLS-terminated HTTPS listeners
# on the gateway
env:
PILOT_ENABLE_QUIC_LISTENERS: true
Install with the above IstioOperator
YAML
$ istioctl apply -f istio-operator.yaml -y
$ istioctl proxy-status
This demo uses the httpbin application under samples/
in istio/istio
repository.
-
Create
httpbin
namespace and label it for istio injection$ kubectl create namespace httpbin $ kubectl label namespace httpbin istio-injection=enabled
-
Deploy
httpbin
application$ kubectl -n httpbin apply -f samples/httpbin/httpbin.yaml
-
Create TLS certificates for the ingress
- Configuration for the certificate
[req] default_bits = 2048 prompt = no distinguished_name = req_distinguished_name req_extensions = san_reqext [ req_distinguished_name ] countryName = IN stateOrProvinceName = KA organizationName = QuicCorp [ san_reqext ] subjectAltName = @alt_names [alt_names] DNS.0 = httpbin.quic-corp.com
- Generate CA and server certificates
$ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:4096 -subj "/C=IN/ST=KA/O=QuicCorp" -keyout quiccorp-ca.key -out quiccorp-ca.crt $ openssl req -out httpbin.csr -newkey rsa:2048 -nodes -keyout httpbin.key -config httpbin.cnf $ openssl x509 -req -days 365 -CA quiccorp-ca.crt -CAkey quiccorp-ca.key -set_serial 0 -in httpbin.csr -out httpbin.crt -extfile httpbin.cnf -extensions san_reqext
- Create a secret in
istio-system
namespace$ kubectl -n istio-system create secret tls httpbin-cred --key=httpbin.key --cert=httpbin.crt
- Configuration for the certificate
Gateway to expose the service (notice that there is no HTTP/3 or QUIC related configs here. The mirror server is created automatically)
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: httpbin-gateway
spec:
selector:
app: istio-ingressgateway
istio: ingressgateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- httpbin.quic-corp.com
tls:
mode: SIMPLE
credentialName: httpbin-cred
For routing to httpbin, we also need a VirtualService
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: httpbin-route
spec:
hosts:
- httpbin.quic-corp.com
gateways:
- httpbin-gateway
http:
- name: httpbin-default-route
route:
- destination:
host: httpbin.httpbin.svc.cluster.local
port:
number: 8000
Now apply, the configs and check if "two" listeners are created at the gateway for the specified port (one TCP and another QUIC listeners).
$ kubectl -n istio-system apply gateway-config.yaml
$ istioctl proxy-config listeners istio-ingressgateway-6fdcddbb8b-9rpkx.istio-system
ADDRESS PORT MATCH DESTINATION
0.0.0.0 8443 SNI: httpbin.quic-corp.com Route: https.443.https.httpbin-gateway.istio-system
0.0.0.0 8443 SNI: httpbin.quic-corp.com Route: https.443.https.httpbin-gateway.istio-system
0.0.0.0 15021 ALL Inline Route: /healthz/ready*
0.0.0.0 15090 ALL Inline Route: /stats/prometheus*
You can examine the config dump further to check for listeners with transport socket of type envoy.extensions.transport_sockets.quic.v3.QuicDownstreamTransport
.
NOTE: For testing HTTP/3 traffic a custom build of curl with HTTP/3 support may be required. In this demo it is called qcurl
. Follow the instructions here.
First, note down the ingress gateway IP address as INGRESS_IP
$ kubectl -n istio-system get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer 10.96.112.53 172.18.200.1 15021:32534/TCP,443:31597/TCP,443:31597/UDP 79m
istiod ClusterIP 10.96.255.123 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 79m
$ export INGRESS_IP=172.18.200.1
Now, send HTTP/2 traffic (-k
flag is used to skip TLS certificate verification. This is for demo purposes only)
$ curl -svk --http2 --resolve httpbin.quic-corp.com:443:$INGRESS_IP https://httpbin.quic-corp.com/headers
[ Output truncated ]
...
> GET /headers HTTP/2
> Host: httpbin.quic-corp.com
> user-agent: curl/7.76.1
> accept: */*
...
< HTTP/2 200
< server: istio-envoy
< date: Wed, 27 Oct 2021 05:05:59 GMT
< content-type: application/json
< content-length: 601
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 1
< alt-svc: h3=":443"; ma=86400
<
{
"headers": {
"Accept": "*/*",
"Host": "httpbin.quic-corp.com",
"User-Agent": "curl/7.76.1",
"X-B3-Parentspanid": "819c2b2cab59bbe8",
"X-B3-Sampled": "0",
"X-B3-Spanid": "0462ac631afb5f84",
"X-B3-Traceid": "3fa44e3bbbcd21a3819c2b2cab59bbe8",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe:https://cluster.local/ns/httpbin/sa/httpbin;Hash=97bf9c90d4a5b9bb8f5da3e825dfa34f04631400420649394741807a320aa0a1;Subject=\"\";URI=spiffe:https://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
}
}
Notice the alt-svc
header in the reply. This may be used by the clients (like Google Chrome) to detect HTTP/3 support at the endpoint.
Now, send HTTP/3 traffic
$ qcurl -svk --http3 --resolve httpbin.quic-corp.com:443:$INGRESS_IP https://httpbin.quic-corp.com/headers
* Added httpbin.quic-corp.com:443:172.18.200.1 to DNS cache
* Hostname httpbin.quic-corp.com was found in DNS cache
* Trying 172.18.200.1:443...
* Connect socket 5 over QUIC to 172.18.200.1:443
* Sent QUIC client Initial, ALPN: h3,h3-29,h3-28,h3-27
* Connected to httpbin.quic-corp.com () port 443 (#0)
* h3 [:method: GET]
* h3 [:path: /headers]
* h3 [:scheme: https]
* h3 [:authority: httpbin.quic-corp.com]
* h3 [user-agent: curl/7.78.0-DEV]
* h3 [accept: */*]
* Using HTTP/3 Stream ID: 0 (easy handle 0xa64260)
> GET /headers HTTP/3
> Host: httpbin.quic-corp.com
> user-agent: curl/7.78.0-DEV
> accept: */*
>
< HTTP/3 200
< server: istio-envoy
< date: Wed, 27 Oct 2021 05:08:46 GMT
< content-type: application/json
< content-length: 642
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 2
< alt-svc: h3=":443"; ma=86400
<
{
"headers": {
"Accept": "*/*",
"Host": "httpbin.quic-corp.com",
"Transfer-Encoding": "chunked",
"User-Agent": "curl/7.78.0-DEV",
"X-B3-Parentspanid": "d9f3d78e3f6ddc5c",
"X-B3-Sampled": "0",
"X-B3-Spanid": "3f6c920b02a92b73",
"X-B3-Traceid": "d5c22cd31ddec119d9f3d78e3f6ddc5c",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Internal": "true",
"X-Forwarded-Client-Cert": "By=spiffe:https://cluster.local/ns/httpbin/sa/httpbin;Hash=97bf9c90d4a5b9bb8f5da3e825dfa34f04631400420649394741807a320aa0a1;Subject=\"\";URI=spiffe:https://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account"
}
}
Visit istio.io to learn how to use Istio.
- Preparing for Development Mac
- Preparing for Development Linux
- Troubleshooting Development Environment
- Repository Map
- GitHub Workflow
- Github Gmail Filters
- Using the Code Base
- Developing with Minikube
- Remote Debugging
- Verify your Docker Environment
- Istio Test Framework
- Working with Prow
- Test Grid
- Code Coverage FAQ
- Writing Good Integration Tests
- Test Flakes
- Release Manager Expectations
- Preparing Istio Releases
- 1.5 Release Information
- 1.6 Release Information
- 1.7 Release Information
- 1.8 Release Information
- 1.9 Release Information
- 1.10 Release Information
- 1.11 Release Information
- 1.12 Release Information
- 1.13 Release Information
- 1.14 Release Information
- 1.15 Release Information
- 1.16 Release Information
- 1.17 Release Information
- 1.18 Release Information
- 1.19 Release Information
- 1.20 Release Information
- 1.21 Release Information
- 1.22 Release Information
- 1.23 Release Information
- 1.24 Release Information
- Collecting Logs and Debug Info
- Dependency FAQ
- Working with discuss.istio.io
- Developing with and hosting upon OpenShift
- Adapter Dev Guide
- Adapter Walkthrough
- Attribute Generating Adapter Walkthrough
- Route Directive Adapter Development Guide
- Out of Tree Adapter Walkthrough
- Running a Local Instance
- Template Dev Guide
- Using a Custom Adapter
- Publishing Adapters and Templates to istio.io
- Enabling Envoy Authorization Service and gRPC Access Log Service With Mixer