Better handling of multiple hosts in VirtualService

[KSmigielski]

Problem

When user specifies more then one host in VirtualService, in envoy configuration there should be only one virutal host with all required domains.

Alternatives

Instead specifying multiple hosts in VirutalService, user can use EnvoyFilter to apply patch which will add custom domain to virtual host.

In my opinion it is problematic, because when application expose more then one port, user will need to patch multiple virtual hosts.

Affected product area

[ ] Ambient
[ ] Docs
[ ] Dual Stack
[ ] Installation
[ ] Networking
[x] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Affected features

[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane

Additional context

We are migrating from our in-house envoy based service-mesh to istio.

We need to add custom host domain in VirtualService for easy migration to Istio from our own envoy based service-mesh. Without that migration will be very problematic for us.

We are fighting with huge difference in memory usage of data plane. We were looking what can cause such problems. And we find out that, if you specify multiple hosts in VirtualService it will create the same number of virtual hosts in envoy. Those virtual hosts are almost identical, the only difference is in name and list of domains.

So we removed additional hosts from VirutalService and memory consumption of istio data-plane drastically decrease.
In small service the difference is 100MB.

For service with */* the difference is 3.5.GB

After that we patched envoy with EnvoyFilter to add custom domains to virtual hosts and we didn't observed any significant changes in memory consumption.

How to reproduce that

You just need to apply VirtualService

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: <service-name>-route
  namespace: <namespace>
spec:
  hosts:
  - <service-name>.<namespace>.svc.cluster.default
  - custom.name
  - other.name
  http:
    - route:
      - destination:
          host: <service-name>.<namespace>.svc.cluster.default

Then just grab config from envoy via istioctl d envoy.

I were using bookinfo example app. And here is example route config with multiple virtual hosts

{
  "version_info": "2024-06-20T13:08:57Z/12",
  "route_config": {
    "@type": "type.googleapis.com/envoy.config.route.v3.RouteConfiguration",
    "name": "9080",
    "virtual_hosts": [
      {
        "name": "custom.name:9080",
        "domains": [
          "custom.name"
        ],
        "routes": [
          {
            "match": {
              "prefix": "/"
            },
            "route": {
              "cluster": "outbound|9080||reviews.default.svc.cluster.local",
              "timeout": "0s",
              "retry_policy": {
                "retry_on": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
                "num_retries": 2,
                "retry_host_predicate": [
                  {
                    "name": "envoy.retry_host_predicates.previous_hosts",
                    "typed_config": {
                      "@type": "type.googleapis.com/envoy.extensions.retry.host.previous_hosts.v3.PreviousHostsPredicate"
                    }
                  }
                ],
                "host_selection_retry_max_attempts": "5",
                "retriable_status_codes": [
                  503
                ]
              },
              "max_grpc_timeout": "0s"
            },
            "metadata": {
              "filter_metadata": {
                "istio": {
                  "config": "/apis/networking.istio.io/v1alpha3/namespaces/default/virtual-service/reviews-route"
                }
              }
            },
            "decorator": {
              "operation": "reviews.default.svc.cluster.local:9080/*"
            }
          }
        ],
        "include_request_attempt_count": true
      },
      {
        "name": "other.name:9080",
        "domains": [
          "other.name"
        ],
        "routes": [
          {
            "match": {
              "prefix": "/"
            },
            "route": {
              "cluster": "outbound|9080||reviews.default.svc.cluster.local",
              "timeout": "0s",
              "retry_policy": {
                "retry_on": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
                "num_retries": 2,
                "retry_host_predicate": [
                  {
                    "name": "envoy.retry_host_predicates.previous_hosts",
                    "typed_config": {
                      "@type": "type.googleapis.com/envoy.extensions.retry.host.previous_hosts.v3.PreviousHostsPredicate"
                    }
                  }
                ],
                "host_selection_retry_max_attempts": "5",
                "retriable_status_codes": [
                  503
                ]
              },
              "max_grpc_timeout": "0s"
            },
            "metadata": {
              "filter_metadata": {
                "istio": {
                  "config": "/apis/networking.istio.io/v1alpha3/namespaces/default/virtual-service/reviews-route"
                }
              }
            },
            "decorator": {
              "operation": "reviews.default.svc.cluster.local:9080/*"
            }
          }
        ],
        "include_request_attempt_count": true
      },
      {
        "name": "reviews.default.svc.cluster.local:9080",
        "domains": [
          "reviews.default.svc.cluster.local",
          "reviews",
          "reviews.default.svc",
          "reviews.default",
          "10.96.140.14"
        ],
        "routes": [
          {
            "match": {
              "prefix": "/"
            },
            "route": {
              "cluster": "outbound|9080||reviews.default.svc.cluster.local",
              "timeout": "0s",
              "retry_policy": {
                "retry_on": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
                "num_retries": 2,
                "retry_host_predicate": [
                  {
                    "name": "envoy.retry_host_predicates.previous_hosts",
                    "typed_config": {
                      "@type": "type.googleapis.com/envoy.extensions.retry.host.previous_hosts.v3.PreviousHostsPredicate"
                    }
                  }
                ],
                "host_selection_retry_max_attempts": "5",
                "retriable_status_codes": [
                  503
                ]
              },
              "max_grpc_timeout": "0s"
            },
            "metadata": {
              "filter_metadata": {
                "istio": {
                  "config": "/apis/networking.istio.io/v1alpha3/namespaces/default/virtual-service/reviews-route"
                }
              }
            },
            "decorator": {
              "operation": "reviews.default.svc.cluster.local:9080/*"
            }
          }
        ],
        "include_request_attempt_count": true
      },
      {
        "name": "allow_any",
        "domains": [
          "*"
        ],
        "routes": [
          {
            "match": {
              "prefix": "/"
            },
            "route": {
              "cluster": "PassthroughCluster",
              "timeout": "0s",
              "max_grpc_timeout": "0s"
            },
            "name": "allow_any"
          }
        ],
        "include_request_attempt_count": true
      }
    ],
    "validate_clusters": false,
    "max_direct_response_body_size_bytes": 1048576,
    "ignore_port_in_host_matching": true
  },
  "last_updated": "2024-06-20T13:08:57.834Z"
}

Possible implementation

I'm not very familiar with istio code, but I briefly checked and it should be very easy to implement.
Here hosts from VirtualService are split based on existence in service registy.
And then here it is used to create virtual host. I think that it just need a little changes to create one virtual host with all required domains.

[KSmigielski]

Do you see any problems for such change?

[howardjohn]

We actually have exact logic like this for gateways:

istio/pilot/pkg/networking/core/gateway.go

Line 579 in 044be69

if !features.EnableRouteCollapse {

but not for mesh. So probably similar logic could be applied