Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML]add InferenceAction request query validation #110147

Conversation

Huaixinww
Copy link
Contributor

When testing Cohere's rerank function, we discovered that if a query is not provided, cohereRerankRequest will throw a NPE.
The validate() method of InferenceAction checks the input but does not validate the query, causing cohereRerankRequest to throw a NPE because it requires the query to be non-null.

Copy link

cla-checker-service bot commented Jun 25, 2024

💚 CLA has been signed

@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v8.15.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Jun 25, 2024
@maxhniebergall maxhniebergall self-assigned this Jun 25, 2024
Copy link
Contributor

@timgrein timgrein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Huaixinww

Thank you for the PR! It would be great, if you could sign the contributor agreement to move forward on the review. For changes in the inference api you can also add the two label :ml and Team:ML to request a review from the ML team.

if (taskType.equals(TaskType.RERANK)) {
if (query == null) {
var e = new ActionRequestValidationException();
e.addValidationError("missing query");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
e.addValidationError("missing query");
e.addValidationError(format("Field [query] cannot be null for task type [%s]", TaskType.RERANK));

}
if (query.isEmpty()) {
var e = new ActionRequestValidationException();
e.addValidationError("query is empty");
Copy link
Contributor

@timgrein timgrein Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also use something like e.addValidationError(format("Field [query] cannot be empty for task type [%s]", TaskType.RERANK)); similar to my comment above

@@ -71,6 +72,84 @@ public void testParsing() throws IOException {
}
}

public void testValidation() throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this test could be split into several smaller tests testValidation_TextEmbedding_Null, testValidation_TextEmbedding_Empty, testValidation_Rerank_Null etc.

@timgrein timgrein added >non-issue :ml Machine learning Team:ML Meta label for the ML team and removed needs:triage Requires assignment of a team area label labels Jun 25, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@Huaixinww
Copy link
Contributor Author

Huaixinww commented Jun 26, 2024

Hi @timgrein
Thank you for your suggestion! I've signed the contributor agreement, formatted the error message, and split the test into several smaller tests.
Please let me know if there are any further improvements or changes needed. Thank you!

Copy link
Member

@davidkyle davidkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thank you for finding the bug and fixing it

@davidkyle
Copy link
Member

@elasticmachine test this please

Copy link
Contributor

@timgrein timgrein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the changes! LGTM 🚢

@timgrein timgrein merged commit 2d45e47 into elastic:main Jun 27, 2024
16 checks passed
@Huaixinww Huaixinww deleted the bugfix/add-InferenceAction-request-query-validation branch August 28, 2024 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external-contributor Pull request authored by a developer outside the Elasticsearch team :ml Machine learning >non-issue Team:ML Meta label for the ML team v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants