Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DRAFT] Initial changes for full-text search in ES|QL #109518

Draft
wants to merge 25 commits into
base: main
Choose a base branch
from
Draft

[DRAFT] Initial changes for full-text search in ES|QL #109518

wants to merge 25 commits into from

Conversation

ChrisHegarty
Copy link
Contributor

@ChrisHegarty ChrisHegarty commented Jun 10, 2024 •
edited by ioanatia
Loading

DRAFT DRAFT DRAFT - not for review

Initial changes for full-text search in ES|QL.

For example:

SEARCH images [
  | WHERE date > now() - 1 month
  | RANK MATCH(scene, "mountain lake")
  | WHERE _score > 0.1
  | LIMIT 100
]
| STATS c = COUNT(votes) BY rating
| LIMIT 5

Non-exhaustive list of outstanding tasks:

  • Review all interactions with current stable code
  • Investigate CSV testing
  • review and iterate on lexer and grammar
  • unit tests for operator
  • flesh out more examples and tests
  • test release and snapshot builds
  • fix issue with filter and match scoring 0.0
  • examine the use of float for _score (encoded into an IntVector)
  • topN on the coordinator
  • float block, and float type BWC
  • support WHERE MATCH
  • ...

@ChrisHegarty @costin
@ioanatia @tteofili
Initial Search in ES|QL changes
c4d3939 
Co-authored-by: ChrisHegarty <[email protected]>
Co-authored-by: Costin Leau <[email protected]>
Co-authored-by: Ioana Tagirta <[email protected]>
Co-authored-by: Tommaso Teofili <[email protected]>
@ChrisHegarty ChrisHegarty added WIP Team:Search Meta label for search team labels Jun 10, 2024
@ChrisHegarty ChrisHegarty changed the title [DRAFT] Initial Search in ES|QL changes [DRAFT] Initial changes for full-text search in ES|QL Jun 10, 2024
ChrisHegarty
ChrisHegarty commented Jun 11, 2024
View reviewed changes
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/parser/StatementParserTests.java Outdated
SEARCH images [
| WHERE date > now() - 1 month
| RANK MATCH(scene, "mountain lake")
| WHERE _score > 0.1
Copy link
Contributor Author

@ChrisHegarty ChrisHegarty Jun 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bogus! find a more general way to do score cut-off

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

take topN, convert to percent, then take ... go figure it out

@ChrisHegarty
Add parenthesized Expression to MATCH (#109584)
2d3d278 
This commit adds parenthesized Expression to MATCH in the esql/search branch.
ChrisHegarty
ChrisHegarty commented Jun 13, 2024
View reviewed changes
...lugin/esql/src/test/java/org/elasticsearch/xpack/esql/parser/SearchStatementParserTests.java
assertStatement(
"""
SEARCH index [
| RANK MATCH(pet, "dog") AND (MATCH(color, "brown") OR MATCH(color, "red")) ]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST / SHOULD / SHOULD NOT match. Check query DSL.

@ChrisHegarty ChrisHegarty mentioned this pull request Jun 25, 2024
Open
17 tasks
@ioanatia ioanatia mentioned this pull request Jul 5, 2024
Merged
@javanna javanna added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch and removed Team:Search Meta label for search team labels Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch WIP
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@ChrisHegarty @javanna @elasticsearchmachine @ioanatia @tteofili