Add unit test for existing point output I/O code

[edwardhartnett]

Pull Request Summary

This PR adds a unit test for the existing points I/O code.

Description

Unit test is added. Also turned on code coverage in the CI.

Please also include the following information:
Once this is merged we can talk about how we want the other CI workflows to run unit tests.

Issue(s) addressed

Part of #682

Commit Message

Add unit test for points I/O code.

Check list

• [ x] Branch is up to date with the authoritative repository (NOAA-EMC) develop branch.
• [ x] Checked the checklist for a developer submitting to develop.
• [x ] If a version number update is required, checked the updating version number checklist.
• [ x] If a new feature was added, a regression test for testing the new feature is added.

Testing

• How were these changes tested?

No code change in this PR, only tests being added for existing code.

This PR includes the contents of #1157

[MatthewMasarik-NOAA]

Thanks @edwardhartnett. I'll start reviewing this.

[MatthewMasarik-NOAA]

Timeline for processing, tomorrow (1/3).

[edwardhartnett]

Once this is merged we can turn on unit tests in the other CI runs. We can also discuss the proliferation of CI workflows and how they might best be organized.

[JessicaMeixner-NOAA]

What are thoughts about moving the model/test directory to under the regression test directory and perhaps renaming it to unit tests so it'd be regtests/unit or regtests/unittests or something similar? I think this is more in line with the current WW3 directory structure.

[edwardhartnett]

I'm happy to move them. @MatthewMasarik-NOAA you agree?

[MatthewMasarik-NOAA]

Hi @edwardhartnett @JessicaMeixner-NOAA, yes that works for me. My vote would be for: regtests/unittests.

[edwardhartnett]

OK I've moved the test directory as suggested.

[MatthewMasarik-NOAA]

OK I've moved the test directory as suggested.

Great! Thank you @edwardhartnett, the directory structure looks good.

[MatthewMasarik-NOAA]

@edwardhartnett

Update: The CI runs on my fork of this branch just came back all successful, so we are good to proceed.

One change: Could we have the new CI test be turned off by default? I see there's an if statement controlling the unit tests with the BUILD_TESTING parameter. Could that be set to false by default, but allow you to override by passing in -DBUILD_TESTING=1? I talked with @JessicaMeixner-NOAA this morning about the Organization Free github plans and how adding CI testing would affect our allotment. The free plan gives us 2000 minutes per month, but that is for all 178 repositories in NOAA-EMC. I tried from the NOAA-EMC github home to see if I could view usage data, but could not. Since we don't know how much we're consuming, but know that we share a fixed budget with 177 other repos, it seems smart to turn this off until we have a solution with respect to allotment. Does that work?

[ukmo-ccbunney]

It's great to see some work towards unit testing in the WW3 code!

[edwardhartnett]

I will turn off GitHub Actions unit testing in this PR, as requested.

@ukmo-ccbunney I agree it would be great to see some unit testing of this code. Programmers work an order-of-magnitude faster on code bases with good unit testing. Nor am I comfortable handing over code to a project that may break it, all unknowing. I like my software to be reliable.

I gave a talk about unit testing NetCDF and NCEPLIBS at the Model Correctness Workshop recently at NCAR. We also presented an AGU poster about adding unit testing to NCEPLIBS.

Adding unit testing to science codes is straightforward - easy programming, in fact. The WW3 code base contains about 260K model code (at least half of which is comments). So I would expect writing full tests would take a good programmer less than a year. In the absence of a dedicated testing programmer, that means if every WW3 programmer wrote tests for all new code, and also pitched in to write tests for some existing code, then WW3 would have unit testing in less than a year.

There must be top-down stakeholder support for unit testing. Without full support from project leadership and senior developers, any such effort will be unsuccessful. Unit testing is a discipline which requires effort and commitment from all programmers and leadership.

@MatthewMasarik-NOAA to further pursue unit testing for WW3, if GitHub is disallowed, we can talk about setting up a Jenkins server on your machine which would run the test code.

Jenkins:

• Free and open source java-based CI system.
• Really nice and easy GUI. Much better than GitHub yml files.
• Can be set up on any system.
• Can be integrated with GitHub with some extra work and some extra tweaks from sysadmins.
• Can run much bigger tests, MPI tests, tests that need a lot of data; it's limited only by your own hardware.
• Does not have to download data or install dependencies for every test run, so much faster.

Every programmer on WW3 should do as I have done: start adding tests for the tasks you are working on. Without programmer effort, this code will never get tests. If neither GitHub nor Jenkins were available, there are still other CI solutions.

The determined programmer can have CI on any project, and the wise programmer is determined to use CI on every project.

[edwardhartnett]

I just installed jenkins on my machine; it took about 10 minutes. Good instructions here: https://phoenixnap.com/kb/install-jenkins-ubuntu#:~:text=installed%20on%20Ubuntu.-,Step%201%3A%20Install%20Java,and%20Java%2011%20on%20Ubuntu.

[MatthewMasarik-NOAA]

I will turn off GitHub Actions unit testing in this PR, as requested.

@ukmo-ccbunney I agree it would be great to see some unit testing of this code. Programmers work an order-of-magnitude faster on code bases with good unit testing. Nor am I comfortable handing over code to a project that may break it, all unknowing. I like my software to be reliable.

I gave a talk about unit testing NetCDF and NCEPLIBS at the Model Correctness Workshop recently at NCAR. We also presented an AGU poster about adding unit testing to NCEPLIBS.

Adding unit testing to science codes is straightforward - easy programming, in fact. The WW3 code base contains about 260K model code (at least half of which is comments). So I would expect writing full tests would take a good programmer less than a year. In the absence of a dedicated testing programmer, that means if every WW3 programmer wrote tests for all new code, and also pitched in to write tests for some existing code, then WW3 would have unit testing in less than a year.

There must be top-down stakeholder support for unit testing. Without full support from project leadership and senior developers, any such effort will be unsuccessful. Unit testing is a discipline which requires effort and commitment from all programmers and leadership.

@MatthewMasarik-NOAA to further pursue unit testing for WW3, if GitHub is disallowed, we can talk about setting up a Jenkins server on your machine which would run the test code.

Jenkins:

* Free and open source java-based CI system.

* Really nice and easy GUI. Much better than GitHub yml files.

* Can be set up on any system.

* Can be integrated with GitHub with some extra work and some extra tweaks from sysadmins.

* Can run much bigger tests, MPI tests, tests that need a lot of data; it's limited only by your own hardware.

* Does not have to download data or install dependencies for every test run, so much faster.

Every programmer on WW3 should do as I have done: start adding tests for the tasks you are working on. Without programmer effort, this code will never get tests. If neither GitHub nor Jenkins were available, there are still other CI solutions.

The determined programmer can have CI on any project, and the wise programmer is determined to use CI on every project.

@edwardhartnett Thank you for the insights on testing. The presentation slides are nice, and can give folks an introduction/refresher to unit testing as we go forward. @JessicaMeixner-NOAA and I have started to have some discussion on incorporating this type of testing, and we've acknowledged this will require a larger discussion for us internally to determine how to proceed.

As we determine our posture, I'm interested to talk more about running jenkins for CI. I believe we could also use a self-hosted runner under github Actions, which is free. Whatever our solution, and I agree we can find a suitable alternative to running on Github if need be, we would need to get it to run on RDHPCS. I'll look forward to talking more offline about this.

I'll include the review next.

[MatthewMasarik-NOAA]

Thanks for making the change to toggle the test to off for now, @edwardhartnett. I re-ran the CI in my fork and the current Intel/GNU builds were successful, and the gnu_io test does not run, so everything functions as expected.

Code review
Pass

Testing
Pass

• io_gnu workflow ran successfully when enabled.
• io_gnu confirmed turned off for now (due to GH Actions quota constraints for Organizations).

[MatthewMasarik-NOAA]

Thanks @edwardhartnett. It's great to see this first checkpoint for the point I/O work, and additionally a first step in the direction of increasing our testing footprint.