Add informational error message for failed loading of nn-ensemble model

[juhoinkinen]

The new error message shows

• the metadata of the model attempted to be loaded, which consists of
  • the Keras version that saved the model,
  • saving timestamp,
• the installed Keras version that attempted to load the model,
• the original error message by Keras.

See for example the last line of the following:

/home/myuser/.cache/pypoetry/virtualenvs/annif-LcTK45Dq-py3.11/lib/python3.11/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator CountVectorizer from version 1.3.0 when using version 1.4.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/home/myuser/.cache/pypoetry/virtualenvs/annif-LcTK45Dq-py3.11/lib/python3.11/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator DecisionTreeClassifier from version 1.3.0 when using version 1.4.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/home/myuser/.cache/pypoetry/virtualenvs/annif-LcTK45Dq-py3.11/lib/python3.11/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator BaggingClassifier from version 1.3.0 when using version 1.4.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
2024-04-16 17:15:02.750040: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 37365600 exceeds 10% of free system memory.
2024-04-16 17:15:02.798074: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 37365600 exceeds 10% of free system memory.
2024-04-16 17:15:02.817460: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 37365600 exceeds 10% of free system memory.
2024-04-16 17:15:02.915322: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 37365600 exceeds 10% of free system memory.
2024-04-16 17:15:02.925984: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 37365600 exceeds 10% of free system memory.
Error: Operation failed backend 'nn_ensemble': loading Keras model from data/projects/kauno-fi/nn-model.keras; model metadata: {'keras_version': '2.13.1', 'date_saved': '2023-11-21@15:47:20'}; you have Keras version 2.15.0. Original error message: "Layer 'dense' expected 2 variables, but received 0 variables during loading. Expected: ['dense/kernel:0', 'dense/bias:0']"

Closes #784.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.64%. Comparing base (1ba7912) to head (5033b4c).
Report is 78 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #785      +/-   ##
==========================================
- Coverage   99.65%   99.64%   -0.01%     
==========================================
  Files          89       91       +2     
  Lines        6405     6817     +412     
==========================================
+ Hits         6383     6793     +410     
- Misses         22       24       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[juhoinkinen]

I'm actually unsure if I'm using the error message prefix formatting correctly or is it not working? Now the beginning part is a bit weird in the message:

Error: Operation failed backend 'nn_ensemble': loading...

[osma]

I'm actually unsure if I'm using the error message prefix formatting correctly or is it not working? Now the beginning part is a bit weird in the message:

Error: Operation failed backend 'nn_ensemble': loading...

I agree it looks weird, but it's formatted here:

Annif/annif/exception.py

Lines 35 to 37 in 1b201dc

	return "{} backend '{}': {}".format(
	self.prefix, self.backend_id, self.message
	)

[juhoinkinen]

The test for the failed reading of metadata from model file does not pass in CICD on Python 3.9, while it passes on Python 3.11 and on my laptop.

The issue is in log capturing:

>       assert "Failed to read metadata from " in caplog.text
E       AssertionError: assert 'Failed to read metadata from ' in ''
E        +  where '' = <_pytest.logging.LogCaptureFixture object at 0x7fd31603c460>.text

tests/test_backend_nn_ensemble.py:236: AssertionError
----------------------------- Captured stderr call -----------------------------
warning: Backend nn_ensemble: Failed to read metadata from tests/data/projects/dummy-en/nonexistent.zip

The log message is not so important to test, so I'll just remove the assert for it.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud