Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/splunkhec] Improve performance and reduce memory consumption #22018

Closed
9 tasks done
dmitryax opened this issue May 16, 2023 · 1 comment
Closed
9 tasks done
Assignees
Labels
enhancement New feature or request exporter/splunkhec

Comments

@dmitryax dmitryax added enhancement New feature or request needs triage New item requiring triage exporter/splunkhec and removed needs triage New item requiring triage labels May 16, 2023
@dmitryax dmitryax self-assigned this May 16, 2023
@atoulme
Copy link
Contributor

atoulme commented May 17, 2023

Performance before and after (comparing between the first PR and the 7th one) (note some benchmarks have changed):

Before:

goos: darwin
goarch: arm64
pkg: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/splunkhecexporter
Benchmark_pushLogData_100_10_10_1024
Benchmark_pushLogData_100_10_10_1024-10     	     410	   2702689 ns/op	 2441942 B/op	   34326 allocs/op
Benchmark_pushLogData_10_100_100_1024
Benchmark_pushLogData_10_100_100_1024-10    	     409	   2672872 ns/op	 2443447 B/op	   34506 allocs/op
Benchmark_pushLogData_10_0_100_1024
Benchmark_pushLogData_10_0_100_1024-10      	     840	   1331842 ns/op	 1226277 B/op	   17175 allocs/op
Benchmark_pushLogData_10_100_0_1024
Benchmark_pushLogData_10_100_0_1024-10      	     836	   1336186 ns/op	 1231659 B/op	   17342 allocs/op
Benchmark_pushLogData_10_10_10_256
Benchmark_pushLogData_10_10_10_256-10       	    2034	    568102 ns/op	  557723 B/op	    7689 allocs/op
Benchmark_pushLogData_10_10_10_1024
Benchmark_pushLogData_10_10_10_1024-10      	    4033	    267946 ns/op	  252992 B/op	    3456 allocs/op
Benchmark_pushLogData_10_10_10_8K
Benchmark_pushLogData_10_10_10_8K-10        	    1746	    687386 ns/op	 1861394 B/op	    2683 allocs/op
Benchmark_pushLogData_10_10_10_1M
Benchmark_pushLogData_10_10_10_1M-10        	    1555	    988593 ns/op	 6039903 B/op	    2684 allocs/op
Benchmark_pushLogData_10_1_1_1024
Benchmark_pushLogData_10_1_1_1024-10        	   41257	     27364 ns/op	   36548 B/op	     351 allocs/op
BenchmarkPushLogRecords
BenchmarkPushLogRecords-10                  	  205251	      5427 ns/op	   31409 B/op	      13 allocs/op
PASS

After

pkg: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/splunkhecexporter
Benchmark_pushLogData_10_10_1024
Benchmark_pushLogData_10_10_1024-10    	   38847	     31386 ns/op	   25167 B/op	     383 allocs/op
Benchmark_pushLogData_10_10_8K
Benchmark_pushLogData_10_10_8K-10      	   13942	     87630 ns/op	   71106 B/op	    1076 allocs/op
Benchmark_pushLogData_10_10_2M
Benchmark_pushLogData_10_10_2M-10      	   13096	     90047 ns/op	   73338 B/op	    1109 allocs/op
Benchmark_pushLogData_10_200_2M
Benchmark_pushLogData_10_200_2M-10     	     631	   1917114 ns/op	 1460391 B/op	   22010 allocs/op
Benchmark_pushLogData_100_200_2M
Benchmark_pushLogData_100_200_2M-10    	      79	  12951741 ns/op	11175612 B/op	  168047 allocs/op
Benchmark_pushLogData_100_200_5M
Benchmark_pushLogData_100_200_5M-10    	      61	  16757523 ns/op	14699264 B/op	  220011 allocs/op
BenchmarkConsumeLogsRejected
BenchmarkConsumeLogsRejected-10        	    1274	    935309 ns/op	  731880 B/op	   11012 allocs/op
PASS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request exporter/splunkhec
Projects
None yet
Development

No branches or pull requests

2 participants