Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

otel-collector v0.35.0 crashing if receiver otlp protocol http is present #4043

Closed
ghost opened this issue Sep 14, 2021 · 6 comments · Fixed by #4044
Closed

otel-collector v0.35.0 crashing if receiver otlp protocol http is present #4043

ghost opened this issue Sep 14, 2021 · 6 comments · Fixed by #4044
Assignees
Labels
bug Something isn't working

Comments

@ghost
Copy link

ghost commented Sep 14, 2021

Describe the bug
Whenever starting the otel-collector version 0.35.0, if the config file contains the following

receivers:
  otlp:
    protocols:
      http:

The start of the collector will fail with the reported below.

Steps to reproduce
Start the otel-collector v 0.35.0 binary using the config file documented below.

What did you expect to see?
I expected to have the otel-collector up and running normally, as this configuration is recommended here:
https://github.com/open-telemetry/opentelemetry-collector/blob/main/receiver/otlpreceiver/README.md

What did you see instead?
The collector failed to start with the following log:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xb29c2f]

goroutine 1 [running]:
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.newConfig({0xc00039b2c0, 0x6, 0x100000000000020})
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/config.go:74 +0x40f
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewHandler({0xe0c340, 0xc000070da0}, {0x0, 0x0}, {0xc0001c9550, 0x4, 0x8})
        go.opentelemetry.io/contrib/instrumentation/net/http/[email protected]/handler.go:71 +0x2b9
go.opentelemetry.io/collector/config/confighttp.(*HTTPServerSettings).ToServer(0xc00007a820, {0xe09760, 0xc00015e300}, {0xc00039ad80, {0xe09ee0, 0xc00038a1c0}, {0x0, 0x0}}, {0xc0001c96a8, 0x1, ...})
        go.opentelemetry.io/collector/config/confighttp/confighttp.go:218 +0x47e
go.opentelemetry.io/collector/receiver/otlpreceiver.(*otlpReceiver).startProtocolServers(0xc00038cb40, {0xe226d8, 0xc00000c6c0})
        go.opentelemetry.io/collector/receiver/otlpreceiver/otlp.go:137 +0x37c
go.opentelemetry.io/collector/receiver/otlpreceiver.(*otlpReceiver).Start(0xc00015e240, {0x1, 0xd0c277}, {0xe226d8, 0xc00000c6c0})
        go.opentelemetry.io/collector/receiver/otlpreceiver/otlp.go:166 +0x29
go.opentelemetry.io/collector/internal/sharedcomponent.(*SharedComponent).Start.func1()
        go.opentelemetry.io/collector/internal/sharedcomponent/sharedcomponent.go:74 +0x3e
sync.(*Once).doSlow(0x7f6568d84108, 0x18)
        sync/once.go:68 +0xd2
sync.(*Once).Do(...)
        sync/once.go:59
go.opentelemetry.io/collector/internal/sharedcomponent.(*SharedComponent).Start(0x258a9b3, {0xe22048, 0xc000032088}, {0xe226d8, 0xc00000c6c0})
        go.opentelemetry.io/collector/internal/sharedcomponent/sharedcomponent.go:73 +0xa5
go.opentelemetry.io/collector/service/internal/builder.(*builtReceiver).Start(...)
        go.opentelemetry.io/collector/service/internal/builder/receivers_builder.go:44
go.opentelemetry.io/collector/service/internal/builder.Receivers.StartAll(0xe04410, {0xe22048, 0xc000032088}, {0xe226a0, 0xc00038db80})
        go.opentelemetry.io/collector/service/internal/builder/receivers_builder.go:73 +0x184
go.opentelemetry.io/collector/service.(*service).Start(0xc00038db80, {0xe22048, 0xc000032088})
        go.opentelemetry.io/collector/service/service.go:109 +0x1fe
go.opentelemetry.io/collector/service.(*Collector).setupConfigurationComponents(0xc000156000, {0xe22048, 0xc000032088})
        go.opentelemetry.io/collector/service/collector.go:228 +0x274
go.opentelemetry.io/collector/service.(*Collector).execute(0xc000156000, {0xe22048, 0xc000032088})
        go.opentelemetry.io/collector/service/collector.go:263 +0x4fd
go.opentelemetry.io/collector/service.New.func1(0xc000382780, {0xcffd0b, 0x2, 0x2})
        go.opentelemetry.io/collector/service/collector.go:117 +0x2e
github.com/spf13/cobra.(*Command).execute(0xc000382780, {0xc000030190, 0x2, 0x2})
        github.com/spf13/[email protected]/command.go:856 +0x60e
github.com/spf13/cobra.(*Command).ExecuteC(0xc000382780)
        github.com/spf13/[email protected]/command.go:974 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/[email protected]/command.go:902
go.opentelemetry.io/collector/service.(*Collector).Run(...)
        go.opentelemetry.io/collector/service/collector.go:146
main.runInteractive({{0xc0004c1860, 0xc0004c19b0, 0xc0004c1920, 0xc0004c1800}, {{0xcff86c, 0x7}, {0xd0c21b, 0x17}, {0xe04410, 0x14}}, ...})
        go.opentelemetry.io/collector/cmd/otelcol/main.go:51 +0xc8
main.run(...)
        go.opentelemetry.io/collector/cmd/otelcol/main_others.go:23
main.main()
        go.opentelemetry.io/collector/cmd/otelcol/main.go:40 +0x1d8

What version did you use?
Version: v0.35.0

What config did you use?
Config:

extensions:
  zpages:

receivers:
  otlp:
    protocols:
      http:

processors:

exporters:
  logging:

service:
  extensions: [zpages]
  pipelines:
    traces:
      receivers:
        - otlp
      processors: []
      exporters:
        - logging

Environment
OS: "Ubuntu 20.04"
Used the binary available here: https://github.com/open-telemetry/opentelemetry-collector/actions/runs/1233371692

@ghost ghost added the bug Something isn't working label Sep 14, 2021
@bogdandrutu
Copy link
Member

Where did you get this version of the collector?

@bogdandrutu
Copy link
Member

@Aneurysm9 this seems to be in otel-go-contrib

@Aneurysm9
Copy link
Member

This appears to result from #4031, which set a MeterProvider on the otelhttp wrapper for the receiver's HTTP server without ensuring that a valid MeterProvider exists. The collector should avoid doing this and the instrumentation configuration should also guard against this.

@codeboten codeboten self-assigned this Sep 14, 2021
@codeboten
Copy link
Contributor

I'll address this issue.

@ghost
Copy link
Author

ghost commented Sep 15, 2021

@bogdandrutu I got it from the test, but I also tried to compile it locally and the same error occurred.
Link to the test: https://github.com/open-telemetry/opentelemetry-collector/actions/runs/1233371692

@codeboten
Copy link
Contributor

@julianocosta89-dt the fix for this has been pushed to main. I've tested the code with the configuration you provided and it appears to be working. Please take a look and re-open if it's not the case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants