Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

stomp client: recover panic: send on closed channel while streaming live build logs #1932

Closed
jmccann opened this issue Feb 7, 2017 · 1 comment · Fixed by #1940
Closed

stomp client: recover panic: send on closed channel while streaming live build logs #1932

jmccann opened this issue Feb 7, 2017 · 1 comment · Fixed by #1940

Comments

@jmccann
Copy link

jmccann commented Feb 7, 2017 •
edited
Loading

Been working with @bradrydzewski on this.

I've been running into the following error randomly:

Feb  3 17:13:39 drone5-server drone5[9888]: time="2017-02-03T17:13:39Z" level=in
fo ip=x.x.x.x latency=1m30.248471505s method=GET path="/ws/logs/repo/name/402/1" status=200 time="2017-02-03T17:13:39Z" user-agent="Mozilla/5.0 (Ma
cintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/5
5.0.2883.95 Safari/537.36"
Feb  3 17:13:39 drone5-server drone5[9888]: 1:M 03 Feb 17:13:39.638 # stomp client: recover panic: send on closed channel
Feb  3 17:13:41 drone5-server drone5[9888]: time="2017-02-03T17:13:41Z" level=info ip=x.x.x.x latency=3.417983ms method=GET path="/api/builds" status=200 time="2017-02-03T17:13:41Z" user-agent="Go-http-client/1.1"

This seems to stem from writing to logs when it is closed @ https://github.com/drone/drone/blob/master/server/stream.go#L76

This does not happen often but as the environment has grown has started to become a daily (or more) occurrence. When it does happen new builds are not processed and a restart of the drone server container is required to get things working again.

Brad may have more info he'd like to attach to this issue as well as I think he understands it better.
@jmccann jmccann mentioned this issue Feb 7, 2017
Closed
@jmccann
Copy link
Author

jmccann commented Feb 7, 2017

Want to call out drone/mq#22 was helpful in tracking this down.

@jmccann jmccann mentioned this issue Feb 14, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't allow "logging" messages after eof is received jmccann/drone
1 participant
@jmccann