Add support for CouchDB clustering

[jvz]

The existing configuration for deploying CouchDB to Kubernetes uses a Deployment type instead of using a higher level concept like StatefulSet which is used in the other clustered services like Kafka, Zookeeper, the controller, the invoker, and anything else that can trivially support stateful sets.

StatefulSet Option

In order to support stateful set based deployments for CouchDB, I found some example code from https://github.com/helm/charts/tree/master/incubator/couchdb which further leads to a small sidecar for publishing K8s DNS records based on CouchDB metadata: https://github.com/kocolosk/couchdb-statefulset-assembler

Alternatively, there appears to be a related PR from the same person upstream to make this a core feature: apache/couchdb#1337. This might be useful to try and help complete for a simpler configuration (i.e., no DNS server sidecar image needed).

Operator Option

Another possible implementation choice would be using something related to https://github.com/nicolai86/couchdb-operator which would abstract things a bit further which simplifies things on our side. This option needs a bit more digging to find out its status as it's almost been two years since its last commit.

Limitations: the linked operator repo has no license. In order to use this, the author will need to be contacted to clarify its licensing terms. See nicolai86/couchdb-operator#1

KubeDB Option

A third option could be to work within the KubeDB project to integrate CouchDB.

[belfhi]

Does the missing support for clustering also mean I can't use a clustered CouchDB that I installed myself? That's exactly the issue I'm having...

[dgrove-oss]

OpenWhisk will work with a clustered CouchDB instance, but you currently need to deploy and initialize the DB yourself. There are some instructions on what to add to your mycluster.yaml to do this here: https://github.com/apache/openwhisk-deploy-kube/blob/master/docs/configurationChoices.md#using-an-external-database

(External includes it running on same kubernetes cluster, just "external" from the perspective of the Helm chart).

[kocolosk]

Hi, I'm not opposed to adding the DNS SRV support to CouchDB directly, but no one is actively working on it. Personally I preferred the use of a "seedlist" config setting for auto-assembling a CouchDB cluster in Kubernetes. See apache/couchdb#1658. This feature enabled us to eliminate the assembler sidecar in the official CouchDB Helm chart at https://hub.helm.sh/charts/couchdb/couchdb.

Another option on the operator front is https://operatorhub.io/operator/couchdb-operator. It's definitely under active development.

[dgrove-oss]

I am going to close this with the rationale that a clustered CouchDB probably means a "real deployment" (ie, not dev sandbox) of OpenWhisk and in such deployments you really should be decoupling the life cycle of the CouchDB instance (which has all of your user's code, activations, etc) from the deployment of the OpenWhisk instance that is executing the actions. Using an external (to the helm chart) CouchDB captures this use case, and is what all of the downstream providers of OpenWhisk as a service are doing (to the best of my knowledge).