Confusing default locations for test_data and sample_data dirs

[pp-mo]

In the module docstring for config.py, we have...

.. py:data:: iris.config.SAMPLE_DATA_DIR

Local directory where sample data exists. Defaults to "sample_data" sub-directory of the Iris package install directory. The sample data directory supports the Iris gallery. Directory contents accessed via :func:`iris.sample_data_path`.

.. py:data:: iris.config.TEST_DATA_DIR

Local directory where test data exists. Defaults to "test_data" sub-directory of the Iris package install directory. The test data directory supports the subset of Iris unit tests that require data. Directory contents accessed via :func:`iris.tests.get_data_path`.

This account does seem to be entirely correct. But of course, those directories don't actually exist in the repo. And If you create them on your disk, they are then in an awkward place cluttering up your repo status.

More confusingly, if you leave these values as provided in "site.cfg.template" (e.g. test_data_dir = /path/to/iris/resources/test_data ), then these are of course non-existent paths.
The config module then spots that those paths are non-existent, but instead of complaining about it, it silently reverts to the aove-mentioned default paths -- even though these are also non-existent.

I don't know if this really makes best sense.
An alternative would be for the defaults to be obviously non-existent directories, like the examples, e.g. "test_data_dir = /test-data-not-defined", and not to automatically replace these with another default. Then it would be clear that you must set these to something sensible for any workable usage, but errors will only happen when you try to actually fetch something from a path based on them.

[ajdawson]

More confusingly, if you leave these values as provided in "site.cfg.template" then these are of course non-existent paths.

I would have thought that the intention of these paths is instructional, as it is a template file the implication is you need to fill in the template. As such I think these seem reasonable.

Of course if you have filled them in with your own paths and the data still cannot be found, perhaps it would be nice to hear about it from iris instead of going to the default location...

[esc24]

I suggest we change the paths in the template to:

test_data_dir = /path/to/test_data
sample_data_dir = /path/to/sample_data

I think this is clearer.

I get the following warning if the dir does not exist:

/home/esc24/iris/lib/iris/config.py:86: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/esc24/iris-test-data/test_data_not_here' is not a valid directory path.

I'm not sure how to get the silent failure @pp-mo indicates.

[ajdawson]

I get the following warning if the dir does not exist:

I didn't check myself, I should have. If there is a warning then I don't think we have a problem. @pp-mo what context did this come up in? I know that if iris can't find sample data you get a unintelligible error from matplotlib when you try to build the docs (problem deep inside the plot directive), but other than that I've never had an issue.

[pp-mo]

My bad, I think I probably missed the message.
I was assisting @dannymet97 with recent work, I think it got lost in the flow somewhere -- there is a warning, I see it now...

[pelson]

Seems like the warning is raised appropriately. We can always improve the documentation of the config values so I'd be 👍 if you submitted a PR. Closing this issue as I wouldn't know what to improve to satisfy the ticket. Feel free to re-open with more specifics.

[pp-mo]

I thought it might be better to complain whenever the result path is non-existent.
But I actually tried this and I think the result is too noisy :

>>> import iris
/home/h05/itpp/git/iris/iris_main/lib/iris/config.py:98: UserWarning: Configured 'sample_data_dir' path, "'/home/h05/itpp/git/iris/iris_main/lib/iris/sample_data'", is not a valid directory path.
  warnings.warn(msg.format(option, path))
/home/h05/itpp/git/iris/iris_main/lib/iris/config.py:95: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/path/to/undefined' is not a valid directory path.
  warnings.warn(msg.format(section, option, c_path))
/home/h05/itpp/git/iris/iris_main/lib/iris/config.py:98: UserWarning: Configured 'test_data_dir' path, "'/home/h05/itpp/git/iris/iris_main/lib/iris/test_data'", is not a valid directory path.
  warnings.warn(msg.format(option, path))
>>> 

So, I think think best left : instead, errors will emerge when using nonexistent paths.