Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Impossible to time out server connections #2352

Closed
1 of 2 tasks
robertpanzer opened this issue Oct 30, 2019 · 0 comments · Fixed by #2353
Closed
1 of 2 tasks

Impossible to time out server connections #2352

robertpanzer opened this issue Oct 30, 2019 · 0 comments · Fixed by #2353

Comments

@robertpanzer
Copy link
Contributor

Issue Type:

  • Bug report
  • Feature request

What happened:
Clients do not correctly close connections, or network components silently drop connections without closing or resetting them.
After several days this can result in Linkerd running out of file handles.

What you expected to happen:
It should be possible to expire server connections.

How to reproduce it (as minimally and precisely as possible):
Start Linkerd with a very low limit of file handles.
Then open as many connections as possible to Linkerd, can be a plain socket, until Linkerd fails to accept the connections due to lack of file handles.

Anything else we need to know?:

Environment:

  • linkerd/namerd version, config files:
  • Platform, version, and config files (Kubernetes, DC/OS, etc): No Kubernetes or such, just plain VMs
  • Cloud provider or hardware configuration:
@robertpanzer robertpanzer mentioned this issue Oct 30, 2019
Merged
adleong pushed a commit that referenced this issue Dec 6, 2019
@robertpanzer @adleong
[#2352] Add configuration server.serverSession to allow expiring serv…
7ef7f25 
…er conne… (#2353)

This PR fixes #2352.

There might be clients opening connections to Linkerd not closing them properly.
Also sometimes network components might silently drop connections to Linkerd.
This results in connections that are never closed and after a longer time this might accumulate to so many open connections until the Linkerd process runs out of file handles.
Right now I could not find a configuration parameter to control expiring of inbound connections like it is supported for client connections.

Therefore this PR proposes to add a configuration parameter `serverSession` to a server that behaves similar to the `clientSession`.
With the following configuration server connections will be closed after 2 seconds of idling or after a full life time of 30 seconds:
```yaml
servers:
- port: 4140
  serverSession:
    idleTimeMs: 2000
    lifeTimeMs: 30000
```

Signed-off-by: Robert Panzer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@robertpanzer