fix: use dhis2/docker-compose instead of amcgee/dhis2-backend as default

[varl]

I forked https://github.com/amcgee/dhis2-backend to https://github.com/dhis2/docker-compose as I figure it's time for the Docker Compose setup to be under the official org on GitHub.

This PR updates the configuration default to the DHIS2 docker-compose repo.

[amcgee]

@varl thanks for pushing this forward! Two questions:

1. Should we move instead of fork? That's what I've done in the past, and it has the added benefit of automatically redirecting any requests to amcgee/dhis2-backend to the new home
2. docker-compose is a bit opaque and maybe (gulp...) we'd want more than one docker-compose setup in the future? We already have one used for e2e tests in dhis2/dhis2-core. What about a name that describes what it's used for, like d2-cluster-docker-compose or cluster-docker-compose or something?

[varl]

1. Sure, that's cool. I'll remove the fork and you can transfer it.

   edit: Done

2. I thought a bit about that, and I think it would be nice to have them all in dhis2/docker-compose E.g.

docker-compose/
├── app-store
│   └── docker-compose.yml
├── cluster
│   └── docker-compose.yml
└── e2e
    └── docker-compose.yml

[amcgee]

2. I thought a bit about that, and I think it would be nice to have them all in dhis2/docker-compose E.g.

Yeah, I think I like that better too, but unfortunately it complicates the technical process of downloading the archive - github only supports downloading and caching whole-repo archives (see dear-github/dear-github#95). We could download the whole thing and then reference different sub-dirs for different cases, but that makes it less generic (harder to host your own compose repo somewhere). Doable though, might be worth it!

For now I think just moving the repo to dhis2/docker-compose will probably be a good first step and then we can add sub-dir support later, I'll do that.

[varl]

Yeah, I think I like that better too, but unfortunately it complicates the technical process of downloading the archive

It does. I do think it might be worth it, and there is the option of forking the dhis2/docker-compose repo, then just removing unused subdirs leaving only cluster/docker-compose.yml, etc.

One variant could be that once we extracted the archive and see that there is no docker-compose.yml in the root of the archive, we can prompt the user for what sub-directory to use.

For example:

d2 cluster up smth
Initializing Docker Compose repository...
Docker Compose repo doesn't have a root docker-compose file...
Select a sub-directory (use `?` for a list): ?
cluster, e2e, custom
Select a sub-directory (use `?` for a list): cluster
Selected: cluster/docker-compose.yml
Spinning up cluster smth

[amcgee]

I've moved amcgee/dhis2-backend to dhis2/docker-compose, for now it's still a single config (flat, no sub-directories).

Thinking about this a little more just now, though, maybe it would make more sense to keep amcgee/dhis2-backend around as the flat option, and support sub-directories in dhis2/docker-compose as well as this PR. I realize that's backtracking a bit on the changes already made to this PR, but I think it will improve the upgrade experience because:

1. Users of the current d2 release will still be able to download the amcgee/dhis2-backend release and have it work
2. Users of the next d2 release triggered by this PR merge (which will be at least a minor bump) will use the dhis2/docker-compose/cluster repo sub-dir.
3. If we migrate the amcgee/dhis2-backend repo to a multi-subdir layout (even after moving it to dhis2/docker-compose) we will inadvertently break older versions of d2

Thought @varl?

[varl]

Sounds good to me!

Edit:

• Will you fork dhis2/docker-compose from dhis2 to your user before we merge chore: move to cluster subdir d2-cluster-docker-compose#4?

• I can implement the subdir in two places, either in cli-helpers-engine or in cli-cluster. I think cli-cluster makes more sense, so I'll do it there.

• It will have to support both subdirs and non-subdirs for now.

[amcgee]

• Will you fork dhis2/docker-compose from dhis2 to your user before we merge dhis2/docker-compose#4?

Yep https://github.com/amcgee/dhis2-backend

• I can implement the subdir in two places, either in cli-helpers-engine or in cli-cluster. I think cli-cluster makes more sense, so I'll do it there.

I think so too

• It will have to support both subdirs and non-subdirs for now.

Why is that?

[amcgee]

Merged dhis2/d2-cluster-docker-compose#4 so we should be ready to add support for subdirs here

[varl]

Why is that?

I might have misunderstood the following quote:

Thinking about this a little more just now, though, maybe it would make more sense to keep amcgee/dhis2-backend around as the flat option, and support sub-directories in dhis2/docker-compose as well as this PR.

I interpreted the part about keeping it around as a flat option as in that it should be an option going forward as well, not just keeping it around for old versions of d2. So a user could use a flat docker-compose repo with d2-cluster as well as a sub-directory based docker-compose repo.

If we are only keeping amcgee/dhis2-backend around as a backwards compatible thing for old versions of d2-cluster, then we can scratch that requirement and only support sub-directory based docker-compose repos going forward.

[amcgee]

@varl got it, yep I just meant for backwards compatibility!

[varl]

@amcgee This is as simple as I could make it. The cluster/ subdir is configurable, but depends on #53 to actually be configurable by a user. It's not something we should expose through a switch, though it is possible to set up clusters using the ~/.config/d2/config.js file, and using that mechanism it is possible to override the docker-compose repo and the subdirectory we look in.

If you set dockerComposeDirectory: '' it should even work with flat docker-compose repositories out of the box... Bonus feature.

[amcgee]

Nice @varl, looks good! Haven't tested but approving based on code review.

There's actually a super-secret hack which means this is configurable on the command line, since the config is resolved from argv. Yargs doesn't whitelist options, so you could technically do d2 cluster up 2.32.0 --dockerComposeRepository <url> --dockerComposeDirectory myCustomDir. I don't actually love the loosey-goosey yargs approach, but it's a bonus (undocumented and unpromoted) feature for now!

[amcgee]

One thing to consider.... this will create duplicates of the full multi-docker-compose repo for every named cluster in the cache, which might be something we should avoid in the future.

[varl]

Yeah, but I'll see if I can handle that in the #53, or, if I cannot, it's not that many duplicated bytes.

[dhis2-bot]

🎉 This PR is included in version 1.2.4 🎉

The release is available on:

• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀