[DevOps]: Research conda nightly builds for latest main

[tomvothecoder]

Is your feature request related to a problem?

A nightly build might be helpful for contributors who want to test the latest main branch of xcdat without having to setup the development environment for this repo.

Describe the solution you'd like

One possible way might be to have a GH Actions job that automatically builds a “latest” version of the package and pushes it to Anaconda whenever there is a new commit on main.

Describe alternatives you've considered

No response

Additional context

@pochedls brought this idea up in our Slack exchange

[durack1]

+2 from me!

[lee1043]

While I think having nightly is convenient for using latest build without manual install in the dev env, but for PMP, I have retracted it when we move PMP to conda-forge channel. I think one of the reasons discussed was that with nightly version becomes not very traceable while reproducibility is important for PMP. As PMP is transitioning to build on xCDAT, version tracking of xCDAT would be important to PMP. I think I would prefer the standard way of version releasing, and if we setup nightly release in addition, I think we might want to keep it internally and not advertise too widely. Just my 2 cents.

[pochedls]

Maybe this is a discussion item..? I am in the pro-nightly build camp, but agree that we could and should hide it so that a new user doesn't accidentally install it as their default xcdat instance.

[rljacob]

Yes dependent packages should use tagged versions, not latest head-of-main.

[tomvothecoder]

Thank you @pochedls, @lee1043, and @rljacob, I converted this to a discussion.

I agree, the use case for a nightly build should be internal testing and not production usage. If the nightly build is incorrectly used for production, it will quickly result in code divergence since we follow continuous deployment/continuous integration of code changes.

If we do have a nightly build, I suggest making it private to select xcdat contributors who want to help test but don't necessarily need a full development environment.

On the other hand, we should encourage contributors to clone the repo and try to submit PRs too! We'll think more about whether or not there is enough value to justify having a nightly build since it involves additional work to manage it.

[tomvothecoder]

I forgot to mention that we have a repository specifically for xcdat validation against CDAT: https://github.com/xCDAT/xcdat_test.

It includes conda env yml files for building envs with the latest main branch or the latest tagged version.
https://github.com/xCDAT/xcdat_test/tree/main/conda-env.

Maybe we should include an environment in the main repo which includes the latest tagged version (aka a production env).

[durack1]

Hi all, I certainly am keen like @pochedls on having a nightly build to test beta changes. I note that we have this setup in CMOR, with the official conda-forge channel at https://anaconda.org/conda-forge/cmor/files (label = main), and the private unofficial channel at https://anaconda.org/PCMDI/cmor/ (labels = nightly, amongst others). This has worked well and means that you have to explicitly set your channel order and defaults to pull down a nightly

@mauzey1 ping as he has been managing these builds

[tomvothecoder]

Thanks @durack1. I now recognize the other use case for a nightly build, which is to test the integration of the latest main branch with other packages (e.g., CMOR, PMP, etc.) to ensure it works. Let me know if that sounds correct.

This is different from my initial thought, which was having a nightly build for individual contributors to test the latest changes directly without having to clone the repo and build the dev env.

A nightly build is still able to achieve both things. Will do some more thinking here.

[durack1]

test the integration of the latest main branch with other packages (e.g., CMOR, PMP, etc.)

@tomvothecoder exactly, so you can "release" a beta, announcing to no one and see how this works across the dependencies. Once you have positive feedback, then more broadly announce it, or add an alphaX or betaX tag and broaden its uptake before formally releasing/announcing an update. As you note, having a nightly means you can pull xcdat=nightly into your favorite hybrid environment and test it with other released packages

[tomvothecoder]

TLDR: We decided to go with rapid release cycles (more frequent releases on conda) over nightly conda builds.

Here are the pros and cons that we listed for nightly conda builds. We found the cons outweighted the pros, and wanted to follow the same convention as most other Python packages.

Pros

• With longer release schedules, the latest version of main can be tested by users in their dev env or developers with their package build workflows to ensure dependencies don't break their environment or package.
• Reduces friction for people to test the latest version of main, since they don't need to clone the repo and setup the dev env

Cons:

• xcdat is relatively lightweight and much less likely to break due to dependency issues, unlike CDAT which has significantly more dependencies to manage
• Overhead of managing another build process and label
  • Most other packages such as xarray, cf_xarray, etc. don’t do nightly builds probably for this reason
• Users might confuse nightly build for a versioned release and keep on using it for production
  • I believe @jasonb5 ran into this problem with users
  • Can be addressed by using another conda label (but introduces overhead mentioned above)
• Users might see a nightly build is broken one day and not realize it has been already fixed the next
  • They would need to integrate a CI/CD process for pulling in the newest nightly builds of a package whenever it is deployed
• We follow CI/CD for rapid development, so PRs are merged quickly on the main branch and we can roll out new releases subsequently.
• Users aren’t able to try out features in development branches until it is merged into main for the nightly build anyways
  • They would need to clone the repo regardless
  • These branches are often long-standing, unlike branches for bug fixes.

Decision

Instead, we are following rapid releases since we follow continuous integration/continuous deployment.

• Minor for features and patch for bug fixes
• Follow same practices of xarray and cf_xarray, and other packages