-
Notifications
You must be signed in to change notification settings - Fork 165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Does anyone have problems in training from scratch with GCNet on ImageNet? #10
Comments
Did you try to finetune the resnet? |
I have the similar problem. When training from scratch, the model converges very slowly. |
Me too |
Sorry for the late reply. At the first place, we don't have enough resources for training from scratch. Noted that we use the same augmentation method as SENet. |
@xvjiarui |
We use the 'add' one by default. On the other hand, 'scale' is similar to SENet. Both of them should not have converge issue. |
@xvjiarui |
Our internal code base is used for classification training. We may try to release a cleaned version in the future. But it is not on schedule yet. The block structure is the same. You could simply add it to your own code. |
Hi! Did you solve the problem? |
@xvjiarui |
For all the yes/no questions, the answers are yes. You are understanding correctly. |
|
Hi, Thanks for your reply. |
Hi, @13952522076 |
Hi, @Shiro-LK |
Thanks a lot for your reply. Looks like not the issue of overfitting (train loss vs. val loss). Anyway, appreciate your work and it helped a lot. 😺 |
@ma-xu Hello, excuse me, has the problem been solved? |
Using the best setting of GC-ResNet50 and train it from scratch on ImageNet, I found it will be stuck in a high loss in the early epochs before the training loss begins to decline normally. Therefore the final result is much lower than original ResNet50. Note that one difference from the original paper is that the GC modules are embedded in each bottleneck exactly as SE does, for a fair comparison.
Does anyone have the same problem?
This may be the case since the authors report the ImageNet results via a finetuning setting, which is not very common when validating models on ImageNet Benchmarks. At least all other modules (SE, SK, BAM, CBAM, AA) are following a training-from-scratch setting.
The text was updated successfully, but these errors were encountered: