feat: add hetzner cloud provider #167

Thunderbottom · 2021-02-15T22:30:06Z

adds a provider for hetzner cloud that does:

filter instances by label selector and datacenter location
fetch public/private IP address based on provided address type
checks for associated floating IPs in case of blocked public/private IP addresses
accepts API token as environment variable: HCLOUD_TOKEN

Enhancement over: #159
Closes: #73

hashicorp-cla · 2021-02-15T22:30:10Z

All committers have signed the CLA.

provider/hcloud/hcloud_discover.go

provider/hcloud/hcloud_discover_test.go

VoyTechnology

LGTM!

svenstaro · 2021-03-30T04:34:14Z

Could we get this merged?

laukaichung · 2021-04-06T17:13:16Z

While waiting for it to be merged, is there any way for now to automatically get a list of private IPs for the retry_join field without hardcoding a list of IPs? I'm planning to run a consul cluster on Hetzner Cloud with private network. It seems that GetPrivateIPs from go-sockaddr, e,g retry_join={{ GetPrivateIPs }} would work but I''m not sure how to use it in the config.

dnephin · 2021-04-06T17:22:42Z

I don't think there is any support for that yet, but hashicorp/consul#9100 is a proposal that should make it possible.

vitamindit · 2021-04-06T17:25:06Z

While waiting for it to be merged, is there any way for now to automatically get a list of private IPs for the retry_join field without hardcoding a list of IPs? ...

I'm waiting for it too :) ... here is my approach:

You get private IP addresses once the server is assigned to a network or subnet.

Then you should use e.g. the hcloud server-info API. This will return the available servers including their private IP addresses for available interfaces.

From this server-info response you can then construct the list of desired private cluster IP addresses.

KevinGimbel · 2021-05-21T21:06:09Z

What's blocking this PR from being merged? I'm currently evaluating Consul for work and I'd love to use/try the auto-discovery feature as part of this PoC.

gardion · 2021-06-17T15:28:19Z

Please merge, it would massively help adopting Consul for our platform.

tomschlenkhoff · 2021-07-07T09:08:26Z

Any help needed? Any blockers left? If not please (with sugar on top) merge, would be a boost for Consul/Hetzner adoption in Germany/EU.

Thunderbottom · 2021-07-07T10:19:46Z

I believe there's no blockers, although the repo hasn't been updated since last year. So I'm not sure if it'll be merged any time soon.

commit 7623bb6 Author: Chinmay D. Pai <[email protected]> Date: Tue Feb 16 03:56:04 2021 +0530 doc: add documentation for hcloud Signed-off-by: Chinmay D. Pai <[email protected]> commit 297a1f5 Author: Chinmay D. Pai <[email protected]> Date: Tue Feb 16 03:54:08 2021 +0530 ci: add tests for hcloud provider * add terraform files for hcloud server setup * add tests for public IPv4, IPv6, and private IPv4 * add test to check for location filter * add circleci entry for hcloud provider Signed-off-by: Chinmay D. Pai <[email protected]> commit 49c8fb3 Author: Chinmay D. Pai <[email protected]> Date: Tue Feb 16 03:51:44 2021 +0530 feat: add hcloud to providers map and update go modules * adds hcloud provider to providers map * adds go module entry for hcloud, updates go mod sum Signed-off-by: Chinmay D. Pai <[email protected]> commit 0ea43fc Author: Chinmay D. Pai <[email protected]> Date: Tue Feb 16 03:43:18 2021 +0530 feat: add hetzner cloud (hcloud) provider adds a provider for hetzner cloud that does: * filter instances by label selector and datacenter location * fetch public/private IP address based on provided address type * checks for associated floating IPs in case of blocked public/private IP addresses * accepts API token as environment variable: HCLOUD_TOKEN Signed-off-by: Chinmay D. Pai <[email protected]>

provider/hcloud/hcloud_discover.go

ronaldburns · 2022-02-27T12:19:16Z

This is a feature we need for our infrastructure. It doesn't appear like it's going to be committed anytime soon.
We've forked this and made a fix or two.

go-discover
consul

If you'd like to immediately use this implementation, binaries have been added for the latest releases.

Consul 1.11.3 (hcloud)

Consul 1.11.4 (hcloud)

Consul 1.12.0-beta1 (hcloud)

devnet-io · 2022-04-02T23:16:40Z

What are the blockers on getting this merged? This will be very useful for my org. Happy to assist with what is needed.

KrakenMare · 2022-04-13T16:00:48Z

Echoing the above. It would be fantastic to see this merged or insight into what is needed for a merge.

pugnacity · 2022-07-28T11:43:21Z

any news here?

marco-m · 2022-11-03T19:30:29Z

Hello @dnephin, @riddhi89, @alvin-huang,
we were almost there with #159 :-), then after a while @Thunderbottom dared to summon the skeleton (just past-Halloween pun) and did this second attempt.
I understand very well that you are fully loaded, but if you could spare a moment to review this PR many of us would be happy. Thanks again!
@Thunderbottom would you have time to resolve the conflicts, to ease the work of the HashiCorp reviewers? Merci!

test/tf/hcloud/versions.tf

go.mod

acaloiaro · 2022-11-24T15:27:02Z

FYI @Thunderbottom I can't imagine you're enthusiastic to do more work on this PR since it hasn't been re-reviewed yet, but in case you are -- a merge conflict has occurred.

Thunderbottom · 2022-11-24T16:27:02Z

Hi, haven't gotten the time to check GitHub that often anymore. I have fixed all the issues and rebased to the latest master. let me know if anything else needs to be done.

svenstaro · 2022-11-24T17:46:45Z

@riddhi89 Any chance we could get this merged?

Neyury · 2023-07-08T21:40:54Z

Any updates?

adds a provider for hetzner cloud that does: * filter instances by label selector and datacenter location * fetch public/private IP address based on provided address type * checks for associated floating IPs in case of blocked public/private IP addresses * accepts API token as environment variable: HCLOUD_TOKEN Signed-off-by: Chinmay D. Pai <[email protected]>

* adds hcloud provider to providers map * adds go module entry for hcloud, updates go mod sum Signed-off-by: Chinmay D. Pai <[email protected]>

* add terraform files for hcloud server setup * add tests for public IPv4, IPv6, and private IPv4 * add test to check for location filter * add circleci entry for hcloud provider Signed-off-by: Chinmay D. Pai <[email protected]>

Signed-off-by: Chinmay D. Pai <[email protected]> Signed-off-by: Chinmay D. Pai <[email protected]>

Signed-off-by: Chinmay D. Pai <[email protected]>

Neyury · 2023-07-12T20:59:06Z

Hello! @alvin-huang any chance you could review this PR? Or do you have somebody else from HashiCorp to suggest? I really want to see Hetzner Cloud in go-discovery so that Сonsul and Nomad could use auto-join. Thanks!

acaloiaro · 2023-07-12T21:19:12Z

@Neyury If you're desperate to use @Thunderbottom's patch, you can do what I did and fork nomad and add a replace to its go.mod as I've done here: hashicorp/nomad@332c3e4

Then my server.server_join stanza looks like:

  server_join {
    retry_join = [ "provider=hcloud label_selector=nomad-server apiToken=${hetzner_compute_access_token}" ]
  }

Note: You'll want to label your nomad server compute nodes with nomad-server to use the above.

I've been running a hetzner nomad cluster this way for over 6 months. I do recommend pinning your fork to a tagged version of nomad, however.

I have a 1.4.x nomad build with patched go-discover here: https://github.com/acaloiaro/nomad/releases/tag/v1.4.x

You really should not use binaries from unknown strangers on the internet. But feel free to use it in a dev/test environment if you feel that it's sufficiently isolated from the rest of your infrastructure

tagirb · 2023-07-14T10:30:14Z

Hi Yuriy, thanks for taking care of this feature, also very interested in it. I'd suggest involving @LKaemmerling from Hetzner for this.

apricote

Julian here from the @hetznercloud Integrations team. This looks fine regarding the usage of hcloud-go, I can not speak about go-discover.

Happy to help if you got any questions.

provider/hcloud/hcloud_discover.go

Co-authored-by: Julian Tölle <[email protected]>

acaloiaro · 2023-08-25T06:12:52Z

Hashicorp are "semi-actively moving our repos to https://github.com/hashicorp/go-netaddr".

I emailed a member of their team to learn about the status of this repository. Those are the words of one individual, but you can see from the project activity that the project is not under heavy development.

Anyone waiting for this or other features to land in go-discover should probably not hold their breath about them being added to go-discover. It sounds like this project has become too brittle and difficult to maintain, motivating the migration to go-netaddr. An understandable move given this project's dependencies.

With that said, I don't see how go-netaddr replaces go-discover's cloud host-aware functionality. go-netaddr seems like a nice complement to, but not replacement for go-discover.

kaspergrubbe · 2024-08-06T18:55:54Z

Hello from 2024, I also hit this in my own setup, and would have loved for go-discover to support hcloud.

VoyTechnology suggested changes Feb 18, 2021

View reviewed changes

provider/hcloud/hcloud_discover.go Outdated Show resolved Hide resolved

provider/hcloud/hcloud_discover.go Outdated Show resolved Hide resolved

provider/hcloud/hcloud_discover_test.go Show resolved Hide resolved

Thunderbottom force-pushed the hcloud-provider branch from 2517c71 to 7623bb6 Compare February 18, 2021 18:15

VoyTechnology approved these changes Feb 18, 2021

View reviewed changes

tomschlenkhoff mentioned this pull request Jul 7, 2021

Add support for Hetzner #73

Open

ronaldburns suggested changes Feb 27, 2022

View reviewed changes

provider/hcloud/hcloud_discover.go Outdated Show resolved Hide resolved

marco-m reviewed Nov 3, 2022

View reviewed changes

test/tf/hcloud/versions.tf Outdated Show resolved Hide resolved

marco-m reviewed Nov 3, 2022

View reviewed changes

go.mod Outdated Show resolved Hide resolved

Thunderbottom force-pushed the hcloud-provider branch from 7623bb6 to 8a7a187 Compare November 24, 2022 16:09

Thunderbottom added 3 commits July 9, 2023 16:23

feat: add hcloud to providers map and update go modules

2924472

* adds hcloud provider to providers map * adds go module entry for hcloud, updates go mod sum Signed-off-by: Chinmay D. Pai <[email protected]>

ci: add tests for hcloud provider

9268c31

* add terraform files for hcloud server setup * add tests for public IPv4, IPv6, and private IPv4 * add test to check for location filter * add circleci entry for hcloud provider Signed-off-by: Chinmay D. Pai <[email protected]>

Thunderbottom added 3 commits July 9, 2023 16:25

doc: add documentation for hcloud

ff7e499

Signed-off-by: Chinmay D. Pai <[email protected]> Signed-off-by: Chinmay D. Pai <[email protected]>

chore: update go mod to use latest hcloud-go lib

055b72a

Signed-off-by: Chinmay D. Pai <[email protected]>

chore: fix api_token reference and update hcloud tf provider

524be32

Signed-off-by: Chinmay D. Pai <[email protected]>

Thunderbottom force-pushed the hcloud-provider branch from 018b71f to 524be32 Compare July 9, 2023 10:58

apricote reviewed Jul 14, 2023

View reviewed changes

provider/hcloud/hcloud_discover.go Outdated Show resolved Hide resolved

chore: identify misbheaving application IDs

91cd52c

Co-authored-by: Julian Tölle <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add hetzner cloud provider #167

feat: add hetzner cloud provider #167

Thunderbottom commented Feb 15, 2021

hashicorp-cla commented Feb 15, 2021 •

edited

Loading

VoyTechnology left a comment

svenstaro commented Mar 30, 2021

laukaichung commented Apr 6, 2021 •

edited

Loading

dnephin commented Apr 6, 2021

vitamindit commented Apr 6, 2021

KevinGimbel commented May 21, 2021

gardion commented Jun 17, 2021

tomschlenkhoff commented Jul 7, 2021

Thunderbottom commented Jul 7, 2021

ronaldburns commented Feb 27, 2022 •

edited

Loading

devnet-io commented Apr 2, 2022

KrakenMare commented Apr 13, 2022

pugnacity commented Jul 28, 2022

marco-m commented Nov 3, 2022

acaloiaro commented Nov 24, 2022

Thunderbottom commented Nov 24, 2022

svenstaro commented Nov 24, 2022

Neyury commented Jul 8, 2023

Neyury commented Jul 12, 2023

acaloiaro commented Jul 12, 2023 •

edited

Loading

tagirb commented Jul 14, 2023 via email •

edited

Loading

apricote left a comment

acaloiaro commented Aug 25, 2023 •

edited

Loading

kaspergrubbe commented Aug 6, 2024

feat: add hetzner cloud provider #167

Are you sure you want to change the base?

feat: add hetzner cloud provider #167

Conversation

Thunderbottom commented Feb 15, 2021

hashicorp-cla commented Feb 15, 2021 • edited Loading

VoyTechnology left a comment

Choose a reason for hiding this comment

svenstaro commented Mar 30, 2021

laukaichung commented Apr 6, 2021 • edited Loading

dnephin commented Apr 6, 2021

vitamindit commented Apr 6, 2021

KevinGimbel commented May 21, 2021

gardion commented Jun 17, 2021

tomschlenkhoff commented Jul 7, 2021

Thunderbottom commented Jul 7, 2021

ronaldburns commented Feb 27, 2022 • edited Loading

If you'd like to immediately use this implementation, binaries have been added for the latest releases.

Consul 1.11.3 (hcloud)

Consul 1.11.4 (hcloud)

Consul 1.12.0-beta1 (hcloud)

devnet-io commented Apr 2, 2022

KrakenMare commented Apr 13, 2022

pugnacity commented Jul 28, 2022

marco-m commented Nov 3, 2022

acaloiaro commented Nov 24, 2022

Thunderbottom commented Nov 24, 2022

svenstaro commented Nov 24, 2022

Neyury commented Jul 8, 2023

Neyury commented Jul 12, 2023

acaloiaro commented Jul 12, 2023 • edited Loading

tagirb commented Jul 14, 2023 via email • edited Loading

apricote left a comment

Choose a reason for hiding this comment

acaloiaro commented Aug 25, 2023 • edited Loading

kaspergrubbe commented Aug 6, 2024

hashicorp-cla commented Feb 15, 2021 •

edited

Loading

laukaichung commented Apr 6, 2021 •

edited

Loading

ronaldburns commented Feb 27, 2022 •

edited

Loading

acaloiaro commented Jul 12, 2023 •

edited

Loading

tagirb commented Jul 14, 2023 via email •

edited

Loading

acaloiaro commented Aug 25, 2023 •

edited

Loading