-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added How To use FastaiLRFinder Guide #11
Conversation
@Priyansi it looks good to me ! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the PR @Priyansi, very nice work! I just have one comment.
how-to-guides/fastai-lr-finder.ipynb
Outdated
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"When attached to the `trainer`, this handler follows the same procedure used by [fastai](https://docs.fast.ai/callback.schedule.html#LRFinder). The model is trained for `num_iter` iterations while the learning rate is increased from its initial value specified by the optimizer to the upper bound called `end_lr`. This increase can be linear (`step_mode=\"linear\"`) or exponential (`step_mode=\"exp\"`). The default `step_mode` is exponential which is recommended for larger learning rate ranges while linear provides good results for small ranges." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
learning rate is increased from its initial value specified by the optimizer to the upper bound called
end_lr
That's not 100% true anymore, we recently added the start_lr option, see here pytorch/ignite#2111
, so I think saying it will increase from start_lr
to end_lr
and if start_lr
is None it will start from LR with optimizer be default would be better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for pointing that out! I'll make the change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Priyansi ! Good work! I left some comments
Maybe we can Add more headers: E.g: |
Thank you for the comments @trsvchn ! I updated the notebook. |
how-to-guides/fastai-lr-finder.ipynb
Outdated
"sidebar: true\n", | ||
"tags:\n", | ||
" - lr finder\n", | ||
" - fastai lr finder\n", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
" - fastai lr finder\n", |
how-to-guides/fastai-lr-finder.ipynb
Outdated
"\n", | ||
" self.model = resnet18(num_classes=10)\n", | ||
" self.model.conv1 = nn.Conv2d(\n", | ||
" 1, 64, kernel_size=(3, 3), stride=(2, 2), padding=(3, 3), bias=False\n", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to pad as 3 if kernel size is 3, should be padding 1.
Let's remove stride as well.
how-to-guides/fastai-lr-finder.ipynb
Outdated
{ | ||
"cell_type": "markdown", | ||
"source": [ | ||
"Let's see how we can achieve better results by using the `FastAILRFinder` handler. But first, let's restore the initial state of the model and optimizer so we can re-train them from scratch. " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Let's see how we can achieve better results by using the `FastAILRFinder` handler. But first, let's restore the initial state of the model and optimizer so we can re-train them from scratch. " | |
"Let's see how we can achieve better results by using the `FastaiLRFinder` handler. But first, let's restore the initial state of the model and optimizer so we can re-train them from scratch. " |
And let's add a link to our docs of FastaiLRFinder
…o add-howto-fastai-lrfinder
Thank you @vfdev-5 for the review, I have made the suggested changes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Fixes #5
Moved FastaiLRFinder_MNIST.ipynb notebook to how-to guides.
Changes in the flow of the notebook, explanation, comments and code were made. ProgressBar handler was removed to avoid unnecessary code.