provide support for backend traffic forwarding

[jbguerraz]

This brings decoupling between vip's address/port and backend's (api server) address/port.
Coupling is setup at start using either iptables or IPVS (depends if load balancer is enabled or not) if the backend address isn't 0.0.0.0 (which is the default value and is backward compatible).

It also adds/removes backends as they become ready/not-ready (in addition to already implemented in / out)

lbBackEndPort looks unused and conflict/confuse with new backendPort that is used to define the api server port. Maybe better to remove ?

rg BackendPort
pkg/kubevip/config_types.go
147:	//BackendPort, is a port that all backends are listening on (To be used to simplify building a list of backends)
148:	BackendPort int `yaml:"backendPort"`

cmd/kube-vip-start.go
39:	kubeVipStart.Flags().IntVar(&startConfigLB.BackendPort, "lbBackEndPort", 6443, "A port that all backends may be using (optional)")

Deprecates lbPort since port is used for the loadbalancer port (and backendPort is used for real (api)servers port)

[thebsdbox]

A few golint checks that have failed.

[chipzoller]

If there are implications for docs that need to be covered, can you either submit a PR in kube-vip/website or raise an issue with the details, please?

[jbguerraz]

If there are implications for docs that need to be covered, can you either submit a PR in kube-vip/website or raise an issue with the details, please?

There are few implications for docs which I updated in this PR but sure, I'll move that to kube-vip/website once this is ready (this is yet a WIP).

[thebsdbox]

This all looks great to me, I have but one concern and I might be completely wrong here 😄. Originally Kube-vip came in a scratch image, meaning that the only thing that lived within the image was the binary itself. This PR moves Kube-vip to use an alpine base meaning that there are now a number of user land tools and "assuming" shell binaries etc. Given that the kube-vip container can mount things like admin.conf is this a security concern 🤔

cc / @neolit123 @yastij @yaocw2020 @detiber (for comment, if they have any)

[neolit123]

K8s distributed images have moved in the direction of being distroless. Preferably there should be no shell and exploitable bits in the image and only the required tools.

Alpine is considered insecure by some providers.

[jbguerraz]

Hello @thebsdbox

Thank you for the review!

I've just added a commit. Sometime CSI init can fail (related: https://bugzilla.redhat.com/show_bug.cgi?id=1817382) at node add time which cause the node status to be not ready (so the node is removed from the IPVS LB pool and k8s API is no more reachable), I had to separate the add and update logic of the node watcher so to always add a new node to the pool (nevermind its status) in case of initial addition to the cluster.

About base image, since this need iptables binary there is not much alternative AFAIK, at least, if eventually there is a security issue, it must be the true for flannel (https://github.com/flannel-io/flannel/blob/master/Dockerfile.amd64) and kube-proxy (https://github.com/kubernetes/kubernetes/blob/master/build/common.sh#L93) as well: we eventually may ask them how they cope with that.

[jbguerraz]

K8s distributed images have moved in the direction of being distroless. Preferably there should be no shell and exploitable bits in the image and only the required tools.

Alpine is considered insecure by some providers.

Hello @neolit123

flannel rely on alpine and kube-proxy on debian. we may move to debian as kube-proxy if preferable.

[neolit123]

I think the debian for kube proxy only contains iptables and other dependencies but no shell or package managers.

[jbguerraz]

I think the debian for kube proxy only contains iptables and other dependencies but no shell or package managers.

 $ docker exec -it k8s_kube-proxy_kube-proxy-qvqc8_kube-system_0af83979-d489-4979-9ce3-0028b644fe32_0 /bin/sh
# apt -v
apt 2.2.4 (amd64)
# 

We can go with the same base image than kube-proxy, I don't mind until we have iptables it's fine.

[jbguerraz]

FTR, another option is to move to nftables (https://github.com/google/nftables) which has a netlink interface so we can stay with scratch then.

[thebsdbox]

Due to security concerns re: userland tools in the image we're looking to move this code to nftables.

[linsite]

You need to git commit with a --signoff to sign off the commit to avoid the DCO check failure. @jbguerraz

[thebsdbox]

This has all been re-implemented with health checks in ipvs, it will likely move to eBPF when we have time.