Query DNS to automatically discover cluster peers

[kocolosk]

We can streamline cluster setup by querying DNS for an SRV record containing the hostnames of other cluster members. Service records are defined in RFC 2782 for this purpose but this is not a strictly academic exercise. As a practical example Kubernetes will create SRV records for members of a StatefulSet. I created a sidecar container that uses those SRV records to automatically join up the members of a StatefulSet here:

https://github.com/kocolosk/couchdb-statefulset-assembler

I think it would be interesting to push that logic down into mem3 directly. I'm envisioning a system that can be configured like so

[mem3]
use_dns_service_discovery = true
service_record_name = _couchdb._tcp.example.net

If a user omits the service_record_name CouchDB would attempt to generate the appropriate record for the query by prepending _couchdb._tcp. to the DNS domain name portion of the host / VM / container FQDN.

[kocolosk]

I think there's an interesting sidebar discussion to be had about whether a node that is configured to use service discovery should surface the current status of that service discovery in e.g. the _up endpoint. It's not uncommon in the Kubernetes case for DNS records to take several seconds to populate, and I think you would not want the database Pods in an initial cluster build to report that they're ready for duty until they've learned of one another's existence.

[janl]

Closing in favour of #1337 (bam!)

[wohali]

I don't like this approach to closing issues that still have unlanded PRs, @janl.

I'd rather have #1337 itself close this issue. If for whatever reason #1337 can't land and is closed, we lose track of the original request by your approach.

[janl]

Suits me, I'd rather not have two things open for one thing here on GitHub because issues and PRs are functionally similar

[wohali]

for the record...that's exactly what happened here. #1337 didn't get merged, and if this stayed closed, we'd have lost the enhancement request entirely. (Which may still happen, given this is nearly 18 months old with no action.)