Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluation Update #365

Merged
merged 5 commits into from
May 5, 2023
Merged

Evaluation Update #365

merged 5 commits into from
May 5, 2023

Conversation

SHUMKASHUN
Copy link
Member

Fix some style problems in PR351.

Copy link
Contributor

@research4pan research4pan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Some further improvement tricks:

src/lmflow/pipeline/evaluator.py

  • If most code of _evaluate_acc_with_accelerator and _evaluate_acc_with_deepspeed are the same, write them together would be a better option, since modifying the accuracy computation logics once can result in support of both modes.
  • [Style] line 224: if want a detailed logging message format, using logger.info instead of print would be better.
  • [Style] line 226: if (self.evaluator_args.use_wandb == True) -> if self.evaluator_args.use_wandb.
  • [Feature] Better raise error for other metrics ppl and nll if accelerator option is specified but not supported.

@research4pan research4pan merged commit c1ce4a3 into main May 5, 2023
@SHUMKASHUN SHUMKASHUN deleted the kashun-evaluation-update branch May 18, 2023 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants