Speed up aggregation

[davidhassell]

Fixes #640, #144

Needs cfunits>=3.3.6

Notes

• constructlist.py - Protect logger calls.
• coordinatereference.py - Protect logger calls.
• flags.py - Protect logger calls.
• auxiliarycoordinate.py - Protect logger calls.
• fielddomain.py - Protect logger calls.
• propertiesdata,py - Protect logger calls. relaxed_units and copy keywords to concatenate.
• propertiesdatabounds,py - Protect logger calls. relaxed_units and copy keywords to concatenate.
• formula_terms.py - Protect logger calls.
• query.py - Protect logger calls.
• read.py - Protect logger calls.
• timeduration.py - Protect logger calls.
• functions.py - Deprecate hash_array now that we have Dask names.
• fieldlist.py - cull_graph defaults to False. relaxed_units and copy keywords to concatenate.
• data.py - Deterministic name for the data; new copy keyword to concatenate; cull_graph defaults to False; add potential short-circuit to equals.
• utils.py - Speed up conform_units
• dimensioncoordinate.py - Speed up _infer_direction.
• field.py - Protect logger call; cull_graph defaults to False. relaxed_units and copy keywords to concatenate.
• netcdfread.py - Don't cache values from field constructs; Unicode updates; fix up element caching.
• umread.py - Protect logger calls; set array datatypes; avoid unnecessary copies; cache regular coordinate arrays; set cached elements on Data objects; don't build stash2standardnames dictionary on import.
• cellmethod.py - Protect logger calls, tweak equivalent.
• aggregate.py - Protect logger calls; Lots of changes that are hopefully sufficiently documented (let me know if not :)). A major change is the relocation of the concatenate commands from def _aggregate_2_fields to def aggregate, for reasons described in the new comments.

[sadielbartholomew]

Leaving an intermediate review, since I've just covered all of the files directly under the cf/ directory (i.e. not within a sub-directory e.g. data, mixin), notably the big one aggregate.py, and it might be useful to give you time to respond to feedback in the meantime. I'll complete the review by the end of the day.

Generally everything looks good, though do note in particular the comment regarding the higher log levels being captured by the protecting keywords. Otherwise just minor comments so far. Thanks.

[sadielbartholomew]

Still mid-review. I've just covered the data directory changes, with some further minor feedback as registered below. I'll try to complete the review tonight.

[sadielbartholomew]

I have covered everything but the tests and documentation changes now. Thought I'd leave some more intermediate comments, whilst you appear to be addressing the previous feedback.

[sadielbartholomew]

Thanks for addressing much feedback so far, David. To aid further review, please can you resolve the conflicts here, since quite a list has developed at this stage...

[davidhassell]

resolve the conflicts

Done.

[davidhassell]

Oh hell - I've mucked it up. Need to revert all of the last changes.

[davidhassell]

I'm not sure why the merge conflicts have disappeared after I reverted their resolutions, but I think everything looks ok ....

[sadielbartholomew]

but I think everything looks ok

The conflict resolution seems fine to me, at this stage.

Just finishing up the review. One question: did you mean to commit the changes under docs other than under docs/source, namely those in docs/_downloads, docs/_images and docs/recipes to touch recipe-related content? I am wondering if that was unintentional since it seems unrelated to this PR...

[davidhassell]

One question: did you mean to commit the changes under docs other than under docs/source, namely those in docs/_downloads, docs/_images and docs/recipes to touch recipe-related content? I am wondering if that was unintentional since it seems unrelated to this PR...

No. Oops. I guess I did something odd and the merged #653 became entrained, somehow. I think it's benign in that it'll all come out in the wash, but if there's a good way of removing these that's also be nice.

[sadielbartholomew]

I think it's benign in that it'll all come out in the wash, but if there's a good way of removing these that's also be nice.

I'll try some git CLI wizardry (or, tomfoolery, if it goes wrong...) to exclude all but docs/source from docs in this PR. If it looks about right I'll push it up, unless you tell me not to (I assume in the worst case I push up something wrong we can recover via your local branch, so it should be safe?)...

[davidhassell]

I'll try some git CLI wizardry

Thanks, Sadie - sounds like a plan,

[sadielbartholomew]

Hi David, all done via using git checkout origin/main -- docs/<directory involving recipes touched> to revert any files under those to the state they are in the current (up-to-date) main branch. That's all pushed in the latest commit but it is easier now just to check the full diff/'Files changed' page and see that no recipes changes now feature (I had a brief check to see it did what I hoped it would). (Sorry for the delay, it took a while to work out how to push to this specific branch of yours, from my own copy. Kept trying to push to a new branch under your user account by mistake and it naturally wouldn't let me do that.)

I'll finish the full PR review by checking the testing, within an hour or so. Thanks.

[sadielbartholomew]

All previous feedback addressed, thanks. I just had the testing to look through to finish the review and I've done that now. It all looks good (though we can remove a skip, as commented in-line). I also did a final sanity check on the state of the PR after the feedback changes and after my push to revert the recipe-related changes that made their way into the PR. All tests pass locally.

All good, please merge!

[davidhassell]

Hi Sadie - OK, ready to merge! Thanks for your review of this - really appreciated.