-
Notifications
You must be signed in to change notification settings - Fork 217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reproducability issue with ChemProt results for relation extraction using SciBERT #112
Comments
Hello, |
@laleye No. I'm still awaiting a reply from the authors about this. I tried hyperparameter tuning but this is the maximum score I've gotten so far. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I'm currently in midst of replicating the results for the relation extraction task on the ChemProt dataset using SciBERT but so far have been unsuccessful in achieving the F1 score as mentioned in your paper. Using the hyperparameters as described in your paper, I've been able to get an F1 score of 0.51 (Test Set) as detailed by the results from running the script provided in your codebase. Please advise if any further hyperparameter tuning or other tweaking is required to reproduce the F1 score as mentioned in your paper.
Thanks in advance!
The text was updated successfully, but these errors were encountered: