[incubator/fluentd-cloudwatch] Fluentd stop processing logs after ~ 10 minutes

[mfornasa]

Is this a request for help?: No

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Version of Helm and Kubernetes: Kubernetes 1.8, Helm 2.6.2

Which chart: incubator/fluentd-cloudwatch

What happened:
Fluentd stops processing logs, but it keeps running. This happens on at least three clusters, including an old one on 1.6, all installed with kops on AWS.

What you expected to happen:
Fluentd should not hang.

How to reproduce it (as minimally and precisely as possible):
Approx 10 minutes after installation of the daemonset on a kops-created Kubernetes cluster, no logs are sent to cloudwatch.

Anything else we need to know:
This is a known issue: fluent/fluentd#1630.
I don't know if this is specific to kops or not, but this basically makes this chart unusable. The fix, according to the ticket, is setting enable_stat_watcher false on the tail inputs.