Cross-validation for local segmentation problem

[davidkvcs]

I see that if I set number_of_cross_validation_splits this will generate a HyperDrive run, however, I only have the option to train locally.
Do you suggest I should train on all of my 835 cases in a single run or is there some way to crossvalidate locally for a segmentation problem?

You can see more about my segmentation problem below.

---

I am implemting InnerEye for tumor segmentation of head and neck cancer patients on PET-CT images.
The training data, which we cannot upload to Azure or other cloud services, consists of 835 annotated PET-CT cases.
I am implementing multiple major openly and freely available methods for my problem, and comparing the performance of these. Inner-Eye is one of them.

I have read your documentation and I have defined a new class for my problem:

from pathlib import Path
from InnerEye.ML.configs.segmentation.HeadAndNeckBase import HeadAndNeckBase

class HNC_tumor_dgk_HeadAndNeckBase(HeadAndNeckBase):
    def __init__(self) -> None:
        super().__init__(
            ground_truth_ids=["tumor"],
            image_channels=["ct", "pet"],
            local_dataset=Path("<my/local/path/to/the/data>"),
            enable_logging_outside_azure_ml=True,
            num_dataload_workers=24,
            max_num_gpus=1
            )

And correspondingly my dataset.csv looks like this:

subject,filePath,channel
1,path/to/case1_000_0000.nii.gz,ct
1,path/to/case1_000_0001.nii.gz,pet
1,path/to/case1_000.nii.gz,tumor
2,path/to/case2_001_0000.nii.gz,ct
2,path/to/case2_001_0001.nii.gz,pet
2,path/to/case2_001.nii.gz,tumor
...
835,path/to/case835_001_0000.nii.gz,ct
835,path/to/case835_001_0001.nii.gz,pet
835,path/to/case835_001.nii.gz,tumor

AB#6004

[ant0nsc]

Hi @davidkvcs , for segmentation models, the InnerEye codebase only supports cross-validation in the cloud. We found the cost of running cross-validation on a single box so prohibitive that the additional code complexity is not worth it.