[telemetryquerylanguage] add AND, OR, and parens to WHERE clauses #12213

kentquirk · 2022-07-10T19:52:56Z

Implements more complex expressions for WHERE clauses in the grammar.

Description:
The transform processor currently accepts a WHERE clause with a single condition, but there are times where multiple conditions are needed in order to filter to the correct data.

This updates the grammar to allow AND and OR with proper precedence, as well as grouping with parentheses.

Note that during development, the parser was restructured to become a standalone component usable by multiple processors, so the code was ported from #11880 (where there were many individual commits).

Link to tracking Issue:
Fixes #10195.
Closes #11880 and supercedes it.

Testing:

The lexer builder was extracted to a separate function
Tests were added for the lexer to ensure that keywords are not confused with other identifiers
Failing tests were added for the parser with complex conditions
New types added to the grammar (Expression, etc) to support more elaborate expressions
Grammar structured to handle precedence correctly
Expression evaluation will be extracted to a more general structure
More lexer tests
Parser tests to confirm proper operation
Add tests for boolean expression evaluators

Documentation:

Grammar extensions documented

TylerHelmuth · 2022-07-11T16:06:17Z

@kentquirk I am going to take an in-depth look at this today but based on my first pass I wanted to say great job. This is a complicated feature and you've modeled it in an understandable way

TylerHelmuth

@kentquirk reviewed in depth. Mostly just nits.

pkg/telemetryquerylanguage/tql/README.md

pkg/telemetryquerylanguage/tql/parser.go

pkg/telemetryquerylanguage/tql/condition.go

pkg/telemetryquerylanguage/tql/boolean_value.go

pkg/telemetryquerylanguage/tql/parser.go

TylerHelmuth · 2022-07-11T19:30:44Z

/cc @anuraaga

kentquirk · 2022-07-12T13:37:03Z

@TylerHelmuth I believe I've addressed all the issues. Thanks for the fantastic feedback.

Also, I believe that I understand from slack that @anuraaga is unavailable right now.

pkg/telemetryquerylanguage/tql/parser.go

pkg/telemetryquerylanguage/tql/boolean_value.go

pkg/telemetryquerylanguage/tql/parser.go

pkg/telemetryquerylanguage/tql/boolean_value.go

pkg/telemetryquerylanguage/tql/lexer_test.go

pkg/telemetryquerylanguage/tql/parser_test.go

TylerHelmuth

One more small documentation nit, otherwise looks great.

pkg/telemetryquerylanguage/tql/README.md

MikeGoldsmith

Looks good 👍🏻

kentquirk · 2022-07-13T13:44:23Z

I've fixed the nit that the linter was complaining about, and ran the global make gotidy, so hopefully this time it'll pass all the checks.

kentquirk · 2022-07-13T14:55:13Z

Or not. This time i tried running make -j2 golint GROUP=other locally, and fixed things until it passed.

Co-authored-by: Tyler Helmuth <[email protected]>

kentquirk · 2022-07-13T18:42:31Z

Rebased after go mod conflict resolution.

EddieEldridge · 2022-07-15T11:09:24Z

Just want to say thanks for doing this! I badly need to use AND and OR in my queries and this will be a great help.

TylerHelmuth · 2022-07-15T14:11:42Z

@EddieEldridge as a heads up, this was added to the TQL package, which the transform processor doesn't depend on yet, but I have a PR open to change this.

Hopefully the PR is merged before the next release. If it isn't, then it'll be another two weeks before the processor has this functionality.

…en-telemetry#12213) * Port changes to new structure * Add tests for the boolean expressions * remove case-insensitivity * Update docs * Fix lexer test * Add changelog info. * Unify source files and add alwaysFalse * Remove EqualFold * OpEq -> OpComparison * Rename a bunch of things * Rename things for more clarity * Some more renames for clarity * Better names for tests * Add test for no-op parens * Remove "more tests" comment * Update pkg/telemetryquerylanguage/tql/README.md Co-authored-by: Tyler Helmuth <[email protected]> * Satisfy the linter * Try again to satisfy the linter. Co-authored-by: Tyler Helmuth <[email protected]>

…2213) * Port changes to new structure * Add tests for the boolean expressions * remove case-insensitivity * Update docs * Fix lexer test * Add changelog info. * Unify source files and add alwaysFalse * Remove EqualFold * OpEq -> OpComparison * Rename a bunch of things * Rename things for more clarity * Some more renames for clarity * Better names for tests * Add test for no-op parens * Remove "more tests" comment * Update pkg/telemetryquerylanguage/tql/README.md Co-authored-by: Tyler Helmuth <[email protected]> * Satisfy the linter * Try again to satisfy the linter. Co-authored-by: Tyler Helmuth <[email protected]>

kentquirk mentioned this pull request Jul 10, 2022

Add AND and OR to the parser #11880

Closed

9 tasks

kentquirk marked this pull request as ready for review July 10, 2022 21:23

kentquirk requested review from a team and TylerHelmuth as code owners July 10, 2022 21:23

github-actions bot assigned dashpole Jul 10, 2022

kentquirk changed the title ~~Add AND, OR, and parens to WHERE clauses.~~ [telemetryquerylanguage] add AND, OR, and parens to WHERE clauses Jul 11, 2022

TylerHelmuth reviewed Jul 11, 2022

View reviewed changes

TylerHelmuth reviewed Jul 12, 2022

View reviewed changes

TylerHelmuth approved these changes Jul 12, 2022

View reviewed changes

pkg/telemetryquerylanguage/tql/README.md Outdated Show resolved Hide resolved

MikeGoldsmith approved these changes Jul 13, 2022

View reviewed changes

kentquirk added 15 commits July 13, 2022 14:39

Port changes to new structure

7f79a93

Add tests for the boolean expressions

f4b8cb1

remove case-insensitivity

9c28c26

Update docs

a2bb6d3

Fix lexer test

0c4e1cb

Add changelog info.

648add0

Unify source files and add alwaysFalse

386cdc4

Remove EqualFold

6b0c3ca

OpEq -> OpComparison

4c0a73e

Rename a bunch of things

c0636bd

Rename things for more clarity

3f0ecf9

Some more renames for clarity

c546cad

Better names for tests

41fda9f

Add test for no-op parens

379d6b9

Remove "more tests" comment

fd01578

kentquirk and others added 3 commits July 13, 2022 14:39

Update pkg/telemetryquerylanguage/tql/README.md

903757f

Co-authored-by: Tyler Helmuth <[email protected]>

Satisfy the linter

9d89a3e

Try again to satisfy the linter.

a9e33b1

kentquirk force-pushed the kent.ands-ors-2 branch from 61d3bf6 to a9e33b1 Compare July 13, 2022 18:42

codeboten approved these changes Jul 14, 2022

View reviewed changes

codeboten merged commit 341e714 into open-telemetry:main Jul 14, 2022

kentquirk deleted the kent.ands-ors-2 branch July 14, 2022 17:33

kentquirk mentioned this pull request Aug 3, 2022

REQUEST: New membership for kentquirk open-telemetry/community#1124

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[telemetryquerylanguage] add AND, OR, and parens to WHERE clauses #12213

[telemetryquerylanguage] add AND, OR, and parens to WHERE clauses #12213

kentquirk commented Jul 10, 2022 •

edited

Loading

TylerHelmuth commented Jul 11, 2022

TylerHelmuth left a comment

TylerHelmuth commented Jul 11, 2022

kentquirk commented Jul 12, 2022 •

edited

Loading

TylerHelmuth left a comment

MikeGoldsmith left a comment

kentquirk commented Jul 13, 2022

kentquirk commented Jul 13, 2022

kentquirk commented Jul 13, 2022

EddieEldridge commented Jul 15, 2022

TylerHelmuth commented Jul 15, 2022

[telemetryquerylanguage] add AND, OR, and parens to WHERE clauses #12213

[telemetryquerylanguage] add AND, OR, and parens to WHERE clauses #12213

Conversation

kentquirk commented Jul 10, 2022 • edited Loading

TylerHelmuth commented Jul 11, 2022

TylerHelmuth left a comment

Choose a reason for hiding this comment

TylerHelmuth commented Jul 11, 2022

kentquirk commented Jul 12, 2022 • edited Loading

TylerHelmuth left a comment

Choose a reason for hiding this comment

MikeGoldsmith left a comment

Choose a reason for hiding this comment

kentquirk commented Jul 13, 2022

kentquirk commented Jul 13, 2022

kentquirk commented Jul 13, 2022

EddieEldridge commented Jul 15, 2022

TylerHelmuth commented Jul 15, 2022

kentquirk commented Jul 10, 2022 •

edited

Loading

kentquirk commented Jul 12, 2022 •

edited

Loading