what's the T in cosine learning rate when training on ImageNet?

[xpain1]

I wanna training the gcnet on ImageNet, but when I finished the linear warmup, the loss almost no drop. I set the learning rate as 0.1 and the Tmax as 10 with CosineAnnealingLR in pytorch, is that anything wrong ? thx

[xvjiarui]

Sorry for the late reply.
Please note that during ImageNet training, we trained for 120 epoch in total, which means Tmax should also be set to 120 if don't consider linear warmup epoch.

[xpain1]

Does the 120 epoch contain fine-tune stage with lr 0.02?

…

在 2019年9月29日，下午3:07，Jerry Jiarui XU ***@***.***> 写道： Sorry for the late reply. Please note that during ImageNet training, we trained for 120 ep in total, which means Tmax should also be set to 120. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#29?email_source=notifications&email_token=AEN6L6OIQVWXAQLN5DTAQO3QMBH2PA5CNFSM4IZYMSN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD73KZKA#issuecomment-536259752>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEN6L6NRRMZSCM2MEBOMAB3QMBH2PANCNFSM4IZYMSNQ>.

[xvjiarui]

As mentioned in #10, we also successfully trained GCNet for ImageNet classification from scratch.
My last reply is under training from scratch setting.
As for fine-tune setting mentioned in the initial paper (which would be update soon), there is no linear warmup during fine-tune stage. So in this case Tmax=40.