This repository has been archived by the owner on Mar 21, 2024. It is now read-only.
Register all models after training, not only Segmentation models. #455
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…aining jobs in PR builds.
ant0nsc
suggested changes
May 11, 2021
melanibe
reviewed
May 12, 2021
| # the current run is a single one. See the documentation of ModelProcessing for more details. | ||
| self.run_inference(self.checkpoint_handler, ModelProcessing.DEFAULT) | ||
|
|
||
| if self.container.generate_report: |
There was a problem hiding this comment.
[Not this PR] in the future we could allow users to define a generate report function for all containers i guess
There was a problem hiding this comment.
We have one for general lightning containers actually, it is called separately in the else part of this if-else.
There was a problem hiding this comment.
Opened an issue to make the report generation codepath more consistent: #457.
ant0nsc
previously approved these changes
May 12, 2021
ant0nsc
approved these changes
May 12, 2021
melanibe
approved these changes
May 12, 2021
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes the codepath so all models trained on AzureML are registered. The codepath previously allowed only segmentation models (subclasses of
SegmentationModelBase) to be registered. Models are registered after a training run or if theonly_register_modelflag is set. Models may be legacy InnerEye config-based models or may be defined using the LightningContainer class.The PR also removes the AzureRunner conda environment. The full InnerEye conda environment is needed to submit a training job to AzureML.
It splits the
TrainHelloWorldAndHelloContainerjob in the PR build into two jobs,TrainHelloWorldandTrainHelloContainer. It adds a pytest markerafter_training_hello_containerfor tests that can be run after training is finished in theTrainHelloContainerjob.This will solve the issue of model registration in #377 and #398.