From a3e18247417c4a8c4f293992c8dfcf71ca3be42b Mon Sep 17 00:00:00 2001 From: Bryan Aguilar Date: Thu, 29 Dec 2022 14:57:18 -0800 Subject: [PATCH 1/4] Add num_workers doc to README --- processor/groupbytraceprocessor/README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/processor/groupbytraceprocessor/README.md b/processor/groupbytraceprocessor/README.md index 91017e9f8269..6a29e9c5d101 100644 --- a/processor/groupbytraceprocessor/README.md +++ b/processor/groupbytraceprocessor/README.md @@ -28,6 +28,7 @@ processors: groupbytrace/2: wait_duration: 10s num_traces: 1000 + num_workers: 2 ``` ## Configuration @@ -38,6 +39,9 @@ The `num_traces` property tells the processor what's the maximum number of trace The `wait_duration` property tells the processor for how long it should keep traces in the internal storage. Once a trace is kept for this duration, it's then released to the next consumer and removed from the internal storage. Spans from a trace that has been released will be kept for the entire duration again. +The `num_workers` property controls how many concurrent workers the processor will use to process traces. This number +should be set equal to the amount of physical processors available to the collector. + ## Metrics The following metrics are recorded by this processor: From 562c0178807ed94ccebaa13b092cd4e40209c500 Mon Sep 17 00:00:00 2001 From: Bryan Aguilar Date: Thu, 12 Jan 2023 11:25:37 -0800 Subject: [PATCH 2/4] Update gbt README with defaults and num_workers descript --- processor/groupbytraceprocessor/README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/processor/groupbytraceprocessor/README.md b/processor/groupbytraceprocessor/README.md index 6a29e9c5d101..81a8109542ed 100644 --- a/processor/groupbytraceprocessor/README.md +++ b/processor/groupbytraceprocessor/README.md @@ -35,12 +35,11 @@ processors: Refer to [config.yaml](./testdata/config.yaml) for detailed examples on using the processor. -The `num_traces` property tells the processor what's the maximum number of traces to keep in the internal storage. A higher `num_traces` might incur in a higher memory usage. +The `num_traces` (default=1,000,000) property tells the processor what's the maximum number of traces to keep in the internal storage. A higher `num_traces` might incur in a higher memory usage. -The `wait_duration` property tells the processor for how long it should keep traces in the internal storage. Once a trace is kept for this duration, it's then released to the next consumer and removed from the internal storage. Spans from a trace that has been released will be kept for the entire duration again. +The `wait_duration` (default=1s) property tells the processor for how long it should keep traces in the internal storage. Once a trace is kept for this duration, it's then released to the next consumer and removed from the internal storage. Spans from a trace that has been released will be kept for the entire duration again. -The `num_workers` property controls how many concurrent workers the processor will use to process traces. This number -should be set equal to the amount of physical processors available to the collector. +The `num_workers` (default=1) property controls how many concurrent workers the processor will use to process traces. ## Metrics From 3db365a7a3dd2faaabbd2b8033d27e52b720acd6 Mon Sep 17 00:00:00 2001 From: Bryan Aguilar Date: Thu, 12 Jan 2023 11:26:28 -0800 Subject: [PATCH 3/4] Update config.go to remove physical processor recomendation --- processor/groupbytraceprocessor/config.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/processor/groupbytraceprocessor/config.go b/processor/groupbytraceprocessor/config.go index 7d9a3787c787..66f5a3b7feeb 100644 --- a/processor/groupbytraceprocessor/config.go +++ b/processor/groupbytraceprocessor/config.go @@ -25,7 +25,7 @@ type Config struct { // Default: 1_000_000. NumTraces int `mapstructure:"num_traces"` - // NumWorkers is a number of workers processing event queue. Should be equal to physical processors number. + // NumWorkers is a number of workers processing event queue. // Default: 1. NumWorkers int `mapstructure:"num_workers"` From d101bb7f9e896be7981f53154375e3964368e658 Mon Sep 17 00:00:00 2001 From: Bryan Aguilar Date: Thu, 26 Jan 2023 11:44:27 -0800 Subject: [PATCH 4/4] Add GOMAXPROCS starting point --- processor/groupbytraceprocessor/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/processor/groupbytraceprocessor/README.md b/processor/groupbytraceprocessor/README.md index 81a8109542ed..f3572b758a6d 100644 --- a/processor/groupbytraceprocessor/README.md +++ b/processor/groupbytraceprocessor/README.md @@ -39,7 +39,8 @@ The `num_traces` (default=1,000,000) property tells the processor what's the max The `wait_duration` (default=1s) property tells the processor for how long it should keep traces in the internal storage. Once a trace is kept for this duration, it's then released to the next consumer and removed from the internal storage. Spans from a trace that has been released will be kept for the entire duration again. -The `num_workers` (default=1) property controls how many concurrent workers the processor will use to process traces. +The `num_workers` (default=1) property controls how many concurrent workers the processor will use to process traces. If you are looking to optimize this value +then using GOMAXPROCS could be considered as a starting point. ## Metrics