We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Every metric has an output_type. I can't find the place where it is accessed.
output_type
@register_metric( metric="acc", higher_is_better=True, output_type=["loglikelihood", "multiple_choice"], aggregation="mean", )
The register metric decorator does not access this from the args.
def register_metric(**args): # TODO: do we want to enforce a certain interface to registered metrics? def decorate(fn): assert "metric" in args name = args["metric"] for key, registry in [ ("metric", METRIC_REGISTRY), ("higher_is_better", HIGHER_IS_BETTER_REGISTRY), ("aggregation", METRIC_AGGREGATION_REGISTRY), ]: if key in args: value = args[key] assert ( value not in registry ), f"{key} named '{value}' conflicts with existing registered {key}!" if key == "metric": registry[name] = fn elif key == "aggregation": registry[name] = AGGREGATION_REGISTRY[value] else: registry[name] = value return fn return decorate
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Every metric has an
output_type
. I can't find the place where it is accessed.The register metric decorator does not access this from the args.
The text was updated successfully, but these errors were encountered: