Added How To use FastaiLRFinder Guide #11
Conversation
|
@Priyansi it looks good to me ! |
There was a problem hiding this comment.
Thanks a lot for the PR @Priyansi, very nice work! I just have one comment.
how-to-guides/fastai-lr-finder.ipynb
Outdated
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "When attached to the `trainer`, this handler follows the same procedure used by [fastai](https://docs.fast.ai/callback.schedule.html#LRFinder). The model is trained for `num_iter` iterations while the learning rate is increased from its initial value specified by the optimizer to the upper bound called `end_lr`. This increase can be linear (`step_mode=\"linear\"`) or exponential (`step_mode=\"exp\"`). The default `step_mode` is exponential which is recommended for larger learning rate ranges while linear provides good results for small ranges." |
There was a problem hiding this comment.
learning rate is increased from its initial value specified by the optimizer to the upper bound called
end_lr
That's not 100% true anymore, we recently added the start_lr option, see here pytorch/ignite#2111
, so I think saying it will increase from start_lr to end_lr and if start_lr is None it will start from LR with optimizer be default would be better.
There was a problem hiding this comment.
Thank you for pointing that out! I'll make the change.
|
Maybe we can Add more headers: E.g: |
|
Thank you for the comments @trsvchn ! I updated the notebook. |
how-to-guides/fastai-lr-finder.ipynb
Outdated
| "sidebar: true\n", | ||
| "tags:\n", | ||
| " - lr finder\n", | ||
| " - fastai lr finder\n", |
There was a problem hiding this comment.
| " - fastai lr finder\n", |
how-to-guides/fastai-lr-finder.ipynb
Outdated
| "\n", | ||
| " self.model = resnet18(num_classes=10)\n", | ||
| " self.model.conv1 = nn.Conv2d(\n", | ||
| " 1, 64, kernel_size=(3, 3), stride=(2, 2), padding=(3, 3), bias=False\n", |
There was a problem hiding this comment.
Why do we need to pad as 3 if kernel size is 3, should be padding 1.
Let's remove stride as well.
how-to-guides/fastai-lr-finder.ipynb
Outdated
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "Let's see how we can achieve better results by using the `FastAILRFinder` handler. But first, let's restore the initial state of the model and optimizer so we can re-train them from scratch. " |
There was a problem hiding this comment.
| "Let's see how we can achieve better results by using the `FastAILRFinder` handler. But first, let's restore the initial state of the model and optimizer so we can re-train them from scratch. " | |
| "Let's see how we can achieve better results by using the `FastaiLRFinder` handler. But first, let's restore the initial state of the model and optimizer so we can re-train them from scratch. " |
And let's add a link to our docs of FastaiLRFinder
…o add-howto-fastai-lrfinder
|
Thank you @vfdev-5 for the review, I have made the suggested changes. |
Fixes #5
Moved FastaiLRFinder_MNIST.ipynb notebook to how-to guides.
Changes in the flow of the notebook, explanation, comments and code were made. ProgressBar handler was removed to avoid unnecessary code.