Skip to content

Global messaging for apps that need to talk to each other.

Notifications You must be signed in to change notification settings

uncvrd/nats-cluster

 
 

Repository files navigation

Deploy App

fly deploy --dockerfile Dockerfile --config fly.[ENVIRONMENT].toml

Clone Machine in different regions

fly machine clone --region [REGION] [MACHINE_ID] -c fly.[ENVIRONMENT].toml

Create Volumes in different regions

fly volumes create foundry_nats_cluster_data -s 1 -r [REGION] -a foundry-nats-cluster

Global NATS Cluster

NATS is an open source messaging backend suited to many use cases and deployment scenarios. We use it for internal communications at Fly. This repo shows how to use it for your application.

This example creates a federated mesh of NATS servers that communicate over the private, encrypted IpV6 network available to all Fly organizations.

Setup

  1. fly launch --no-deploy

    You'll be prompted for an app name. Hit return to let Fly generate an app name for you. Pick your target organizatio and a starting region.

  2. flyctl deploy

    This will start NATS with a single node in your selected region.

  3. Add more regions with flyctl regions add <region> or

    For this demo, we set ord, syd, cdg regions.

fly regions set ord syd cdg
  1. Scale the application so it can place nodes in the regions.
fly scale count 3

Then run flyctl logs and you'll see the virtual machines discover each other.

2020-11-17T17:31:07.664Z d1152f01 ord [info] [493] 2020/11/17 17:31:07.646272 [INF] [fdaa:0:1:a7b:abc:21de:af5f:2]:4248 - rid:1 - Route connection created
2020-11-17T17:31:07.713Z 21deaf5f cdg [info] [553] 2020/11/17 17:31:07.704807 [INF] [fdaa:0:1:a7b:81:d115:2f01:2]:34902 - rid:19 - Route connection created
2020-11-17T17:31:08.123Z 82fabc30 syd [info] [553] 2020/11/17 17:31:08.114852 [INF] [fdaa:0:1:a7b:81:d115:2f01:2]:4248 - rid:7 - Route connection created
2020-11-17T17:31:08.259Z d1152f01 ord [info] [493] 2020/11/17 17:31:08.241644 [INF] [fdaa:0:1:a7b:b92:82fa:bc30:2]:45684 - rid:2 - Route connection created

Testing the cluster

While the cluster is only accessible from inside the Fly network, you can use Fly's Wireguard support to create a VPN into your Fly organisation and private network.

Then you can use tools such as natscli to subscribe to topics, publish messages to topics and perform various tests on your NATS cluster. Install the tool first.

Once installed, create a context that points at your NATS cluster:

nats context add fly.demo --server appname.internal:4222 --description "My Cluster" --select

You can subscribe to a topic with nats sub topicname:

nats sub fly.demo

And then, in another terminal sessions, we can use nats pub topicname to send either simple messages to that topic:

nats pub fly.demo "Hello World"

Or send multiple messages:

nats pub fly.demo "fly.demo says {{.Cnt}} @ {{.TimeStamp}}" --count=10

You're ready to start integrating NATS messaging into your other Fly applications.

What to try next

  1. NATS streaming offers persistence features, you can create a NATS streaming app by modifying this demo and adding volumes: flyctl volume create

  2. Create a NATS super cluster let you join multiple NATS clusters with gateways. If you want to run regional clusters, you can query the Fly DNS service to with <region>.<app-name>.internal to find server in specific regions.

Discuss

You can discuss this example (and the paired 6pn-demo-chat example) on the dedicated Fly Community topic.

About

Global messaging for apps that need to talk to each other.

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 95.6%
  • Dockerfile 4.4%