Single Network Multi Primary Ambient installation

[moore-nathan]

Currently trying to test out creating a single network Multi-Primary setup with Istio Ambient mode. Using version Istio version 1.22.0 and trying to do as much as possible in Helm. This is deployed into AWS EKS on kubernetes version 1.29.
I currently got it working so that the ztunnel picks up the workloads from each cluster, but there is no balancing between the two clusters. When going through verify-installation only the deployment inside the cluster is getting hit (e.g. in cluster1 only hitting helloworldv1). Was hoping someone could provide assistance as I've gone through all the troubleshooting guides with no luck.

Followed the plugin-ca-cert guide before installing so both clusters have a cacerts secret.
After installing via Helm followed the commands in enbale-endpoint-discovery for adding in the cluster contexts.

istioctl experimental ztunnel-config workloads command

NAMESPACE    POD NAME               IP  NODE    WAYPOINT PROTOCOL
default      metrics-server-xyz     xx  xx      None     TCP
istio-system istiod-xyz             xx  xx      None     TCP
istio-system istiod-xyz             xx  xx      None     TCP
istio-system ztunnel-xyz            xx  xx      None     TCP
istio-system ztunnel-xyz            xx  xx      None     TCP
istio-system ztunnel-xyz            xx  xx      None     TCP
istio-system ztunnel-xyz            xx  xx      None     TCP
istio-system ztunnel-xyz            xx  xx      None     TCP
istio-system ztunnel-xyz            xx  xx      None     TCP
kube-system  coredns-xyz            xx  xx      None     TCP
kube-system  coredns-xyz            xx  xx      None     TCP
kube-system  coredns-xyz            xx  xx      None     TCP
kube-system  coredns-xyz            xx  xx      None     TCP
sample       helloworld-v1-xyz      xx  xx      None     HBONE
sample       helloworld-v2-xyz      xx  xx      None     HBONE
sample       sleep-xyz              xx  xx      None     HBONE
sample       sleep-xyz              xx  xx      None     HBONE

Chart.yaml

---
apiVersion: v2
name: istio-ambient
version: 1.0.0
appVersion: 1.22.0
dependencies:
  - name: base
    alias: istio-base
    version: 1.22.0
    repository: https://istio-release.storage.googleapis.com/charts
  - name: istiod
    version: 1.22.0
    repository: https://istio-release.storage.googleapis.com/charts
  - name: cni
    version: 1.22.0
    repository: https://istio-release.storage.googleapis.com/charts
  - name: ztunnel
    version: 1.22.0
    repository: https://istio-release.storage.googleapis.com/charts

values.yaml

global:
  variant: distroless

enable:
  istiod: true

# Overrides for the `istiod-remote` dep
istiod:
  profile: ambient
  global:
    externalIstiod: true
    meshid: nmoore-test-mesh
    network: network1

# Overrides for the `ztunnel` dep
ztunnel:
  profile: ambient

# Overrides for the `cni` dep
cni:
  profile: ambient

values.cluster1.yaml

istiod:
  global:
    multiCluster:
      clusterName: test-multi-cluster-1

ztunnel:
  multiCluster:
    clusterName: test-multi-cluster-1

values.cluster2.yaml

istiod:
  global:
    multiCluster:
      clusterName: test-multi-cluster-2

ztunnel:
  multiCluster:
    clusterName: test-multi-cluster-2

[howardjohn]

Just FYI - Ambient multicluster is currently not at "beta" quality, only the single cluster. unfortunately this missed some of the docs and was only added recently, so sorry if it was missed (now at https://preliminary.istio.io/latest/docs/releases/feature-stages/#ambient); I think it was just in the blog initially.

So I am not sure the current state exactly, but it is somewhat expected it is lower quality and hasn't received as much testing or attention yet.

[moore-nathan]

Thank you and that is a good note to keep in mind. I was basing the functionality off of this issue #42137 which seemed to suggest this is possible and wanted to see if this holds true