0.13-dev: Accessing prometheus endpoint crashes process

[bismarck]

I'm trying to access the prometheus metrics endpoint, but this is causing the fluent-bit process to crash. I'm able to access all other HTTP endpoints exposed by fluent-bit.

Here's the configuration I am running:

[Fluent Bit]
    Edition		Community Edition
    Version		0.13.0
    Built Flags		 JSMN_PARENT_LINKS JSMN_STRICT FLB_HAVE_TLS FLB_HAVE_SQLDB FLB_HAVE_BUFFERING FLB_HAVE_METRICS FLB_HAVE_HTTP_SERVER FLB_HAVE_FLUSH_LIBCO FLB_HAVE_SYSTEMD FLB_HAVE_FORK FLB_HAVE_PROXY_GO FLB_HAVE_JEMALLOC JEMALLOC_MANGLE FLB_HAVE_REGEX FLB_HAVE_C_TLS FLB_HAVE_ACCEPT4 FLB_HAVE_INOTIFY

[Operating System]
    Name		Linux
    Release		4.4.110-k8s
    Version		#1 SMP Fri Jan 5 16:59:17 UTC 2018

[Hardware]
    Architecture	x86_64
    Processors		1

[Built Plugins]
    Inputs              cpu mem kmsg tail proc disk systemd netif dummy head exec health serial stdin tcp mqtt lib forward random syslog 
    Filters             grep stdout kubernetes parser record_modifier 
    Outputs             counter es exit file forward http influxdb kafka kafka-rest nats null plot stdout td lib flowcounter 

[SERVER] Runtime configuration
    Flush		1
    Daemon		Off
    Log_Level		Info

[INPUT] Instance
    Name		tail.0 (tail, id=0)
    Flags		DYN_TAG 
    Threaded		No
    Tag			kube.*
    Mem_Buf_Limit	4.8M
    Path                /var/log/containers/*.log
    Parser              docker
    Refresh_Interval    5
    Skip_Long_Lines     On
    DB                  /var/log/containers.db

[INPUT] Instance
    Name		tail.1 (tail, id=1)
    Flags		DYN_TAG 
    Threaded		No
    Tag			audit
    Mem_Buf_Limit	4.8M
    Path                /var/log/kube-apiserver-audit.log
    Parser              audit
    Skip_Long_Lines     On
    DB                  /var/log/kube-apiserver-audit.db

[INPUT] Instance
    Name		tail.2 (tail, id=2)
    Flags		DYN_TAG 
    Threaded		No
    Tag			kube-etcd
    Mem_Buf_Limit	4.8M
    Path                /var/log/etcd.log
    Parser              etcd
    Skip_Long_Lines     On
    DB                  /var/log/etcd.db

[INPUT] Instance
    Name		tail.3 (tail, id=3)
    Flags		DYN_TAG 
    Threaded		No
    Tag			kube-proxy
    Mem_Buf_Limit	4.8M
    Path                /var/log/kube-proxy.log
    Multiline           On
    Multiline_Flush     5
    Parser_Firstline    glog_multiline
    Skip_Long_Lines     On
    DB                  /var/log/kube-proxy.db

[INPUT] Instance
    Name		tail.4 (tail, id=4)
    Flags		DYN_TAG 
    Threaded		No
    Tag			kube-apiserver
    Mem_Buf_Limit	4.8M
    Path                /var/log/kube-apiserver.log
    Multiline           On
    Multiline_Flush     5
    Parser_Firstline    glog_multiline
    Skip_Long_Lines     On
    DB                  /var/log/kube-apiserver.db

[INPUT] Instance
    Name		tail.5 (tail, id=5)
    Flags		DYN_TAG 
    Threaded		No
    Tag			kube-controller-manager
    Mem_Buf_Limit	4.8M
    Path                /var/log/kube-controller-manager.log
    Multiline           On
    Multiline_Flush     5
    Parser_Firstline    glog_multiline
    Skip_Long_Lines     On
    DB                  /var/log/kube-controller-manager.db

[INPUT] Instance
    Name		tail.6 (tail, id=6)
    Flags		DYN_TAG 
    Threaded		No
    Tag			kube-scheduler
    Mem_Buf_Limit	4.8M
    Path                /var/log/kube-scheduler.log
    Multiline           On
    Multiline_Flush     5
    Parser_Firstline    glog_multiline
    Skip_Long_Lines     On
    DB                  /var/log/kube-scheduler.db

[INPUT] Instance
    Name		systemd.0 (systemd, id=0)
    Flags		DYN_TAG 
    Threaded		No
    Tag			systemd.*
    Mem_Buf_Limit	4.8M
    Path                /var/log/journal
    Systemd_Filter      _SYSTEMD_UNIT=docker.service
    DB                  /var/log/docker.db

[INPUT] Instance
    Name		systemd.1 (systemd, id=1)
    Flags		DYN_TAG 
    Threaded		No
    Tag			systemd.*
    Mem_Buf_Limit	4.8M
    Path                /var/log/journal
    Systemd_Filter      _SYSTEMD_UNIT=kubelet.service
    DB                  /var/log/kubelet.db

[FILTER] Instance
    Name		kubernetes.0 (kubernetes, id=0)
    Match		kube.*

[OUTPUT] Instance
    Name		es.0 (es, mask_id=1)
    Match		kube.*
    TLS Active		No
    Retry Limit		no limit
    Host.TCP_Port	9200
    Host.Name		es-proxy
    Logstash_Format     On
    Include_Tag_Key     On
    Tag_Key             tag

[OUTPUT] Instance
    Name		es.1 (es, mask_id=2)
    Match		audit
    TLS Active		No
    Retry Limit		no limit
    Host.TCP_Port	9200
    Host.Name		es-proxy
    Logstash_Format     On
    Logstash_Prefix     audit
    Include_Tag_Key     On
    Tag_Key             tag

[OUTPUT] Instance
    Name		es.2 (es, mask_id=4)
    Match		kube-*
    TLS Active		No
    Retry Limit		no limit
    Host.TCP_Port	9200
    Host.Name		es-proxy
    Logstash_Format     On
    Include_Tag_Key     On
    Tag_Key             tag

[OUTPUT] Instance
    Name		es.3 (es, mask_id=8)
    Match		systemd.*
    TLS Active		No
    Retry Limit		no limit
    Host.TCP_Port	9200
    Host.Name		es-proxy
    Logstash_Format     On
    Include_Tag_Key     On
    Tag_Key             tag

[edsiper]

thanks for reporting this bug. I have placed the proper fix in GIT master.

I've also updated our test Docker image to 0.13-dev:0.6:

https://github.com/fluent/fluent-bit-kubernetes-logging/tree/0.13-dev