[pkg/telemetryquerylanguage] Move otel functions to telemetryquerylan…

…guage pkg (open-telemetry#12754)

* Add otel functions to telemetryquerylanguage

pkg/telemetryquerylanguage/functions/tqlotel/README.md

@@ -0,0 +1,143 @@
+# OpenTelemetry Functions
+
+The following functions are intended to be used in implementations of the Telemetry Query Language that interact with otel data via the collector's internal data model, [pdata](https://github.com/open-telemetry/opentelemetry-collector/tree/main/pdata). These functions may make assumptions about the types of the data returned by Paths.
+
+Factory Functions
+- [SpanID](#spanid)
+- [TraceID](#traceid)
+
+Functions
+- [delete_key](#delete_key)
+- [delete_matching_keys](#delete_matching_keys)
+- [keep_keys](#keep_keys)
+- [truncate_all](#truncate_all)
+- [limit](#limit)
+- [replace_all_matches](#replace_all_matches)
+- [replace_all_patterns](#replace_all_patterns)
+
+## SpanID
+
+`SpanID(bytes)`
+
+The `SpanID` factory function returns a `pdata.SpanID` struct from the given byte slice.
+
+`bytes` is a byte slice of exactly 8 bytes.
+
+Examples:
+
+- `SpanID(0x0000000000000000)`
+
+## TraceID
+
+`TraceID(bytes)`
+
+The `TraceID` factory function returns a `pdata.TraceID` struct from the given byte slice.
+
+`bytes` is a byte slice of exactly 16 bytes.
+
+Examples:
+
+- `TraceID(0x00000000000000000000000000000000)`
+
+## delete_key
+
+`delete_key(target, key)`
+
+The `delete_key` function removes a key from a `pdata.Map`
+
+`target` is a path expression to a `pdata.Map` type field. `key` is a string that is a key in the map. 
+
+The key will be deleted from the map.
+
+Examples:
+
+- `delete_key(attributes, "http.request.header.authorization")`
+- `delete_key(resource.attributes, "http.request.header.authorization")`
+
+## delete_matching_keys
+
+`delete_matching_keys(target, pattern)`
+
+The `delete_matching_keys` function removes all keys from a `pdata.Map` that match a regex pattern.
+
+`target` is a path expression to a `pdata.Map` type field. `pattern` is a regex string.
+
+All keys that match the pattern will be deleted from the map.
+
+Examples:
+
+- `delete_key(attributes, "http.request.header.authorization")`
+- `delete_key(resource.attributes, "http.request.header.authorization")`
+
+## keep_keys
+
+`keep_keys(target, keys...)`
+
+The `keep_keys` function removes all keys from the `pdata.Map` that do not match one of the supplied keys.
+
+`target` is a path expression to a `pdata.Map` type field. `keys` is a slice of one or more strings. 
+
+The map will be changed to only contain the keys specified by the list of strings.
+
+Examples:
+
+- `keep_keys(attributes, "http.method")`
+- `keep_keys(resource.attributes, "http.method", "http.route", "http.url")`
+
+## truncate_all
+
+`truncate_all(target, limit)`
+
+The `truncate_all` function truncates all string values in a `pdata.Map` so that none are longer than the limit.
+
+`target` is a path expression to a `pdata.Map` type field. `limit` is a non-negative integer.
+
+The map will be mutated such that the number of characters in all string values is less than or equal to the limit. Non-string values are ignored.
+
+Examples:
+
+- `truncate_all(attributes, 100)`
+- `truncate_all(resource.attributes, 50)`
+
+## limit
+
+`limit(target, limit)`
+
+The `limit` function reduces the number of elements in a `pdata.Map` to be no greater than the limit.
+
+`target` is a path expression to a `pdata.Map` type field. `limit` is a non-negative integer. 
+
+The map will be mutated such that the number of items does not exceed the limit. Which items are dropped is random.
+
+Examples:
+
+- `limit(attributes, 100)`
+- `limit(resource.attributes, 50)`
+
+## replace_all_matches
+
+`replace_all_matches(target, pattern, replacement)`
+
+The `replace_all_matches` function replaces any matching string value with the replacement string.
+
+`target` is a path expression to a `pdata.Mao` type field. `pattern` is a string following [filepath.Match syntax](https://pkg.go.dev/path/filepath#Match). `replacement` is a string. 
+
+Each string value in `target` that matches `pattern` will get replaced with `replacement`. Non-string values are ignored.
+
+Examples:
+
+- `replace_all_matches(attributes, "/user/*/list/*", "/user/{userId}/list/{listId}")`
+
+## replace_all_patterns
+
+`replace_all_patterns(target, regex, replacement)`
+
+The `replace_all_matches` function replaces any segments in a string value that match the regex pattern with the replacement string.
+
+`target` is a path expression to a `pdata.Map` type field. `regex` is a regex string indicating a segment to replace. `replacement` is a string. 
+
+If one or more sections of `target` match `regex` they will get replaced with `replacement`.
+
+Examples:
+
+- `replace_all_patterns(attributes, "/account/\\d{4}", "/account/{accountId}")`

pkg/telemetryquerylanguage/functions/tqlotel/func_delete_key.go

@@ -0,0 +1,35 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http:https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package tqlotel // import "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/telemetryquerylanguage/functions/tqlotel"
+
+import (
+ "go.opentelemetry.io/collector/pdata/pcommon"
+
+ "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/telemetryquerylanguage/tql"
+)
+
+func DeleteKey(target tql.Getter, key string) (tql.ExprFunc, error) {
+ return func(ctx tql.TransformContext) interface{} {
+ val := target.Get(ctx)
+ if val == nil {
+ return nil
+ }
+
+ if attrs, ok := val.(pcommon.Map); ok {
+ attrs.Remove(key)
+ }
+ return nil
+ }, nil
+}

pkg/telemetryquerylanguage/functions/tqlotel/func_delete_key_test.go

@@ -0,0 +1,138 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http:https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package tqlotel
+
+import (
+ "testing"
+
+ "github.com/stretchr/testify/assert"
+ "go.opentelemetry.io/collector/pdata/pcommon"
+
+ "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/telemetryquerylanguage/tql"
+ "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/telemetryquerylanguage/tql/tqltest"
+)
+
+func Test_deleteKey(t *testing.T) {
+ input := pcommon.NewMap()
+ input.InsertString("test", "hello world")
+ input.InsertInt("test2", 3)
+ input.InsertBool("test3", true)
+
+ target := &tql.StandardGetSetter{
+ Getter: func(ctx tql.TransformContext) interface{} {
+ return ctx.GetItem()
+ },
+ }
+
+ tests := []struct {
+ name string
+ target tql.Getter
+ key string
+ want func(pcommon.Map)
+ }{
+ {
+ name: "delete test",
+ target: target,
+ key: "test",
+ want: func(expectedMap pcommon.Map) {
+ expectedMap.Clear()
+ expectedMap.InsertBool("test3", true)
+ expectedMap.InsertInt("test2", 3)
+ },
+ },
+ {
+ name: "delete test2",
+ target: target,
+ key: "test2",
+ want: func(expectedMap pcommon.Map) {
+ expectedMap.Clear()
+ expectedMap.InsertString("test", "hello world")
+ expectedMap.InsertBool("test3", true)
+ },
+ },
+ {
+ name: "delete nothing",
+ target: target,
+ key: "not a valid key",
+ want: func(expectedMap pcommon.Map) {
+ expectedMap.Clear()
+ expectedMap.InsertString("test", "hello world")
+ expectedMap.InsertInt("test2", 3)
+ expectedMap.InsertBool("test3", true)
+ },
+ },
+ }
+ for _, tt := range tests {
+ t.Run(tt.name, func(t *testing.T) {
+ scenarioMap := pcommon.NewMap()
+ input.CopyTo(scenarioMap)
+
+ ctx := tqltest.TestTransformContext{
+ Item: scenarioMap,
+ }
+
+ exprFunc, _ := DeleteKey(tt.target, tt.key)
+ exprFunc(ctx)
+
+ expected := pcommon.NewMap()
+ tt.want(expected)
+
+ assert.Equal(t, expected, scenarioMap)
+ })
+ }
+}
+
+func Test_deleteKey_bad_input(t *testing.T) {
+ input := pcommon.NewValueString("not a map")
+ ctx := tqltest.TestTransformContext{
+ Item: input,
+ }
+
+ target := &tql.StandardGetSetter{
+ Getter: func(ctx tql.TransformContext) interface{} {
+ return ctx.GetItem()
+ },
+ Setter: func(ctx tql.TransformContext, val interface{}) {
+ t.Errorf("nothing should be set in this scenario")
+ },
+ }
+
+ key := "anything"
+
+ exprFunc, _ := DeleteKey(target, key)
+ exprFunc(ctx)
+
+ assert.Equal(t, pcommon.NewValueString("not a map"), input)
+}
+
+func Test_deleteKey_get_nil(t *testing.T) {
+ ctx := tqltest.TestTransformContext{
+ Item: nil,
+ }
+
+ target := &tql.StandardGetSetter{
+ Getter: func(ctx tql.TransformContext) interface{} {
+ return ctx.GetItem()
+ },
+ Setter: func(ctx tql.TransformContext, val interface{}) {
+ t.Errorf("nothing should be set in this scenario")
+ },
+ }
+
+ key := "anything"
+
+ exprFunc, _ := DeleteKey(target, key)
+ exprFunc(ctx)
+}

pkg/telemetryquerylanguage/functions/tqlotel/func_delete_matching_keys.go

@@ -0,0 +1,44 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http:https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package tqlotel // import "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/telemetryquerylanguage/functions/tqlotel"
+
+import (
+ "fmt"
+ "regexp"
+
+ "go.opentelemetry.io/collector/pdata/pcommon"
+
+ "github.com/open-telemetry/opentelemetry-collector-contrib/pkg/telemetryquerylanguage/tql"
+)
+
+func DeleteMatchingKeys(target tql.Getter, pattern string) (tql.ExprFunc, error) {
+ compiledPattern, err := regexp.Compile(pattern)
+ if err != nil {
+ return nil, fmt.Errorf("the regex pattern supplied to delete_matching_keys is not a valid pattern: %w", err)
+ }
+ return func(ctx tql.TransformContext) interface{} {
+ val := target.Get(ctx)
+ if val == nil {
+ return nil
+ }
+
+ if attrs, ok := val.(pcommon.Map); ok {
+ attrs.RemoveIf(func(key string, _ pcommon.Value) bool {
+ return compiledPattern.MatchString(key)
+ })
+ }
+ return nil
+ }, nil
+}