Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suffix properties file name with pod name #309

Closed
wants to merge 2 commits into from

Conversation

solsson
Copy link
Contributor

@solsson solsson commented Mar 8, 2020

Because I'd like to explore ways to transition to static configuration, rather than generating config at init. A completely static config would for be more transparent and could, with for example Kustomize's configmap hashes, support rolling updates better.

While investigating if broker ID could be an --override instead I found two blockers:

  • Oddly the label that was introduced for statefulsets (after this repo's inception) sets the pod name, not the ordinal.
  • Zookeeper needs to be 1-indexed.

I had this comment in code when I was amazed over the design with pod-name instead of pod-ordinal (the former is trivial to create with env expansion):

+++ b/kafka/50kafka.yml
@@ -30,6 +30,9 @@ spec:
         - name: POD_NAME
           valueFrom:
             fieldRef:
+              # Is there any benefit to using this field over metadata.name? fieldPath: metadata.labels['statefulset.kubernetes.io/pod-name']
+              # https://github.com/kubernetes/community/pull/147#issuecomment-350783043
+              # https://github.com/kubernetes/kubernetes/pull/68719 -> new env with fieldPath: metadata.labels['statefulset.kubernetes.io/pod-ordinal']
               fieldPath: metadata.name

It looks like pod-ordinal didn't make it into 1.18.

Experimental, based on the assumption that things like broker.rack
and listener addresses can be set using --overide flags with ev expansion.

If we had the statefulset por-ordinal label we could set those things too,
but now we need to rely on the pod name (+ for zookeeper ID_OFFSET).
@solsson
Copy link
Contributor Author

solsson commented Apr 21, 2020

I added bc2896a because the static files have to be generated with a known number of replicas.

@solsson
Copy link
Contributor Author

solsson commented Apr 22, 2020

I've also introduced a REPLICAS env. It'll need to be on all zookeeper containers wherever we deviate from the hard coded default scale. It's used to pick a properties file, but also to trigger restart on existing zookeeper pods if you scale up or down. You always have to edit both replicas: and this new env. Previously if you tried to do scale or edit the replicas number you'd get an inconsistent zookeeper setup.

It might be that some of the variants are broken now.

@solsson
Copy link
Contributor Author

solsson commented Apr 23, 2020

Now part of #311

@solsson solsson closed this Apr 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant