Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add CyclicalCosineDecay #1599

Merged

Conversation

littletomatodonkey
Copy link
Collaborator

  1. Add CyclicalCosineDecay. The strategy can be seen in : https://arxiv.org/pdf/2012.12645.pdf. In ocr, without any fine tune on hyper-parameters, it can directly takes a 0.3% accuracy gain for CRNN(with 10% more training epochs).
  2. modify the epoch num, because if we want to save models per 5epochs and total epoch is set as 50, then the 50th epoch model will not be saved for the code before(the last epoch is actually 49).

@@ -0,0 +1,49 @@
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个文件不需要吧

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

类似于框架中的lr_scheduler.py,和ocr中封装的learning_rate分开,感觉会更清楚一点

@littletomatodonkey littletomatodonkey merged commit de3e2e7 into PaddlePaddle:dygraph Dec 29, 2020
@littletomatodonkey littletomatodonkey deleted the dyg/add_swa branch November 13, 2021 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants