Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Make unit tests faster #516

Open
dumbledad opened this issue Jun 28, 2021 · 1 comment
Open

Make unit tests faster #516

dumbledad opened this issue Jun 28, 2021 · 1 comment

Comments

@dumbledad
Copy link
Contributor

dumbledad commented Jun 28, 2021

At present running all the unit tests in WSL on my laptop takes 42 minutes. That is too long for me to run all the tests locally before pushing any changes I have made. The unit tests would be more useful if they ran so fast that I would routinely run them all locally while I work on a PR.

The tests that run for more than a minute (in WSL on my XPS15) are:

394.35s call Tests/AfterTraining/test_after_training.py::test_submit_for_inference[True]
207.80s call Tests/SSL/test_ssl_containers.py::test_innereye_ssl_container_cifar10_resnet_simclr
127.20s call Tests/SSL/test_data_modules.py::test_innereye_vision_module
123.78s call Tests/ML/utils/test_checkpoint_handling.py::test_get_best_checkpoint_single_run
62.47s call Tests/ML/utils/test_checkpoint_handling.py::test_get_recovery_path_train_single_run
62.26s call Tests/ML/utils/test_checkpoint_handling.py::test_download_checkpoints_from_single_run
62.14s call Tests/ML/utils/test_checkpoint_handling.py::test_get_checkpoints_to_test_single_run
61.99s call Tests/ML/test_download_upload.py::test_download_recovery_single_run

AB#4166

@dumbledad
Copy link
Contributor Author

Note that 11 fail locally which pass in our Azure build pipeline:

FAILED Tests/AfterTraining/test_after_training.py::test_submit_for_inference[False] - azureml.exceptions._azureml_exc...
FAILED Tests/AfterTraining/test_after_training.py::test_submit_for_inference[True] - azureml.exceptions._azureml_exce...
FAILED Tests/AfterTraining/test_after_training.py::test_recovery_on_2_nodes - azureml.exceptions._azureml_exception.S...
FAILED Tests/ML/test_download_upload.py::test_download_azureml_dataset - NotImplementedError: Linux distribution ubun...
FAILED Tests/ML/test_lightning_containers.py::test_create_fastmri_container - ModuleNotFoundError: No module named 'f...
FAILED Tests/ML/test_lightning_containers.py::test_run_fastmri_container - ValueError: Model name FastMriOnRandomData...
FAILED Tests/ML/visualizers/test_plot_cross_validation.py::test_add_comparison_data - Exception: Run refs_pull_444_me...
FAILED Tests/ML/visualizers/test_visualize_patches.py::test_visualize_patch_sampling[True] - AssertionError: Image si...
FAILED Tests/ML/visualizers/test_visualize_patches.py::test_visualize_patch_sampling_2d - AssertionError: Image sizes...
FAILED Tests/SSL/test_data_modules.py::test_innereye_vision_module - NameError: name 'cv2' is not defined
FAILED Tests/SSL/test_ssl_containers.py::test_innereye_ssl_container_cifar10_resnet_simclr - NameError: name 'cv2' is...

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant