-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add data limitation #255
Add data limitation #255
Conversation
if max_num_examples > 0: | ||
error.value_check( | ||
"<NLP77627434E>", | ||
len(train_stream) <= max_num_examples, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll probably need to make sure a release with caikit/caikit#553 is made and bumped here to make sure we don't run into the same len
problem as #223?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep, thats definitely required
max_num_examples = get_config().training_data_limit.get(cls.MODULE_ID, {}).get(base_model_name, -1) | ||
|
||
if max_num_examples > 0: | ||
error.value_check( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe good to have a tiny unit test case with temp config? :)
Signed-off-by: gkumbhat <[email protected]>
Signed-off-by: gkumbhat <[email protected]>
Signed-off-by: gkumbhat <[email protected]>
Signed-off-by: gkumbhat <[email protected]>
Signed-off-by: gkumbhat <[email protected]>
6c0c249
to
7c79e18
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM but we should probably not merge until caikit bump
Signed-off-by: gkumbhat <[email protected]>
Signed-off-by: gkumbhat <[email protected]>
Signed-off-by: gkumbhat <[email protected]>
Signed-off-by: gkumbhat <[email protected]>
@@ -14,7 +14,7 @@ classifiers=[ | |||
"License :: OSI Approved :: Apache Software License" | |||
] | |||
dependencies = [ | |||
"caikit[runtime-grpc,runtime-http]>=0.22.0,<0.23.0", | |||
"caikit[runtime-grpc,runtime-http]>=0.23.2,<0.25.0", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose chances can be taken that 0.24 won't be breaking for this library
Description