-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Apply FReLU to MobileNetV3 #4
Comments
We use gaussian initialization with std=0.01. I simply replace relu with frelu and it shows a slight improvement (0.1~0.3). We note that MobileNetV3 is a NAS-searched optimal CNN architecture, once you change the architecture (frelu has an additional dw-conv), you might need to search again on this new architecture to achieve the optimal result. |
Thanks for your quick reply. I'll try your suggestions. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I reimplemented FReLU in PyTorch, and apply it on MobileNetV3 by replacing all the hswish with frelu. The ImageNet accuracy is as follow:
My code:
Am I missing something important? As for the gaussian initialization in FReLU, what's the std?
The text was updated successfully, but these errors were encountered: