CI: Download cached remote files in benchmarks.yml

[weiji14]

Description of proposed changes

Make the benchmark GitHub Actions workflow more reliable by pre-downloading some cached files.

Current benchmark tests at 94e4598 takes 12min26s to run. In this PR, it also takes 12min26s 😅

Patches #2908.

Reminders

• Run make format and make check to make sure the code follows the style guide.
• Add tests for new features or tests that would have caught the bug that you're fixing.
• Add new public functions/methods/classes to doc/api/index.rst.
• Write detailed docstrings for all functions/methods.
• If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
• If adding new functionality, add an example to docstrings or tutorials.
• Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash commands are:

• /format: automatically format and lint the code
• /test-gmt-dev: run full tests on the latest GMT development version

[codspeed-hq]

CodSpeed Performance Report

Merging #2923 will improve performances by 8.13%

_{Comparing pytest/mark-benchmark-part-2 (7657005) with main (fbc4e97)}

Summary

⚡ 1 improvements
✅ 63 untouched benchmarks

Benchmarks breakdown

	Benchmark	main	pytest/mark-benchmark-part-2	Change
⚡	test_grdsample_dataarray_out	109.7 ms	101.4 ms	+8.13%

[weiji14]

Hmm, the benchmark CI actually got slower 😅 Running the benchmarks themselves consistently takes 10m30s, but the setup with the cache download adds a few seconds...

[weiji14]

Maybe we can just close this PR. CodSpeed should only be counting CPU cycles according to https://codspeed.io/blog/pinpoint-performance-regressions-with-ci-integrated-differential-profiling, and not execution time. I.e. we won't removing any variance in the benchmarking due to network latency.

[seisman]

Maybe we can just close this PR. CodSpeed should only be counting CPU cycles according to https://codspeed.io/blog/pinpoint-performance-regressions-with-ci-integrated-differential-profiling, and not execution time. I.e. we won't removing any variance in the benchmarking due to network latency.

But tests may fail due to unstable internet connections, so caching is still useful.

[weiji14]

Mm, true, let me reopen this then.

[weiji14]

Btw, I'm seeing lots of warnings at in the benchmarks CI at https://github.com/GenericMappingTools/pygmt/actions/runs/7335964733/job/19974594560#step:8:178 that look like this:

  pygmt/tests/test_basemap.py::test_basemap
    /usr/share/miniconda/lib/python3.12/site-packages/_pytest/python.py:198: PytestReturnNotNoneWarning: Expected None, but pygmt/tests/test_basemap.py::test_basemap returned <pygmt.figure.Figure object at 0x33085dd0>, which will be an error in a future version of pytest.  Did you mean to use `assert` instead of `return`?
      warnings.warn(

Looking at matplotlib/pytest-mpl#183, it seems like installing pytest-mpl should fix it, but we already install it at this line:

pygmt/.github/workflows/benchmarks.yml

Line 64 in 94e4598

geopandas pytest pytest-benchmark pytest-mpl

So not sure what's up with that warning. I also tried reproducing this locally, but can't seem to get the same PytestReturnNotNoneWarning for some reason.

[weiji14]

Gonna ignore the PytestReturnNotNoneWarning mentioned at #2923 (comment) for now, but we'll need to fix it properly before pytest 8.0 comes out.