Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BFCL] Sanity check should be optional and by default off #486

Open
ShishirPatil opened this issue Jun 25, 2024 · 1 comment
Open

[BFCL] Sanity check should be optional and by default off #486

ShishirPatil opened this issue Jun 25, 2024 · 1 comment

Comments

@ShishirPatil
Copy link
Owner

For BFCL eval, the sanity check of weather all RESTful APIs are active should be optional flag that is off by default. It is currently on by default.
@HuanzhiMao
Copy link
Collaborator

HuanzhiMao commented Jun 26, 2024 •
edited
Loading

I think it should be on by default. We want to warn the users as soon as possible if the API is down, because that will make the result inaccurate. The user can then choose to ignore the warning and proceed with the evaluation if they want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ShishirPatil @HuanzhiMao