in_tail: new db.sync property (fluent#396)

When tailing files in environments with high load, likely a database
file is required. By default the database engine (built-in SQLite3)
have a 'FULL' synchronization to disk which means: for every change
flush changes to disk. In some hardware this I/O have performance
penalties.

The following patch introduce a new configuration property called
db.sync which aims to expose the SQLite3 desired sync configuration,
the property accepts one of the following values:

- EXTRA: synchronous is like FULL with the addition that the directory
         containing a rollback journal is synced after that journal is
         unlinked to commit a transaction in DELETE mode. EXTRA
         provides additional durability if the commit is followed closely
         by a power loss.

- FULL (default): When synchronous is FULL (2), the SQLite database
         engine will use the Sync method of the VFS to ensure that all
         content is safely written to the disk surface prior to continuing.
         This ensures that an operating system crash or power failure
         will not corrupt the database. FULL synchronous is very safe,
         but it is also slower. FULL is the most commonly used
         synchronous setting when not in WAL mode.

- NORMAL: When synchronous is NORMAL, the SQLite database engine will
          still sync at the most critical moments, but less often than
          in FULL mode. There is a very small (though non-zero) chance
          that a power failure at just the wrong time could corrupt
          the database in NORMAL mode. But in practice, you are more
          likely to suffer a catastrophic disk failure or some other
          unrecoverable hardware fault.

- OFF:    With synchronous OFF (0), SQLite continues without syncing
          as soon as it has handed data off to the operating system.
          If the application running SQLite crashes, the data will
          be safe, but the database might become corrupted if the
          operating system crashes or the computer loses power before
          that data has been written to the disk surface. On the other
          hand, commits can be orders of magnitude faster with
          synchronous OFF.

note: params description copied from SQLite site

Signed-off-by: Eduardo Silva <[email protected]>

in_tail: new db.sync property (fluent#396)

When tailing files in environments with high load, likely a database
file is required. By default the database engine (built-in SQLite3)
have a 'FULL' synchronization to disk which means: for every change
flush changes to disk. In some hardware this I/O have performance
penalties.

The following patch introduce a new configuration property called
db.sync which aims to expose the SQLite3 desired sync configuration,
the property accepts one of the following values:

- EXTRA: synchronous is like FULL with the addition that the directory
         containing a rollback journal is synced after that journal is
         unlinked to commit a transaction in DELETE mode. EXTRA
         provides additional durability if the commit is followed closely
         by a power loss.

- FULL (default): When synchronous is FULL (2), the SQLite database
         engine will use the Sync method of the VFS to ensure that all
         content is safely written to the disk surface prior to continuing.
         This ensures that an operating system crash or power failure
         will not corrupt the database. FULL synchronous is very safe,
         but it is also slower. FULL is the most commonly used
         synchronous setting when not in WAL mode.

- NORMAL: When synchronous is NORMAL, the SQLite database engine will
          still sync at the most critical moments, but less often than
          in FULL mode. There is a very small (though non-zero) chance
          that a power failure at just the wrong time could corrupt
          the database in NORMAL mode. But in practice, you are more
          likely to suffer a catastrophic disk failure or some other
          unrecoverable hardware fault.

- OFF:    With synchronous OFF (0), SQLite continues without syncing
          as soon as it has handed data off to the operating system.
          If the application running SQLite crashes, the data will
          be safe, but the database might become corrupted if the
          operating system crashes or the computer loses power before
          that data has been written to the disk surface. On the other
          hand, commits can be orders of magnitude faster with
          synchronous OFF.

note: params description copied from SQLite site

Signed-off-by: Eduardo Silva <[email protected]>

filter_grep: use internal regex engine instead of regex.h (fluent#502)

the old implementaion of filter_grep uses the system regex.h
engine instead of libonigmo which is the default solution.

This patch deprecate the usage of regex.h.

note: this plugin should behave as Fluentd grep plugin, it needs
to be fixed to work with rules in 'pipe' mode instead of a simple
ACL.

Signed-off-by: Eduardo Silva <[email protected]>

out_kafka: new config rdkafka.x to configure rdkafka internals

Signed-off-by: Eduardo Silva <[email protected]>

http_server: v1: prometheus metrics: fix usage of flb_sds_cat() (flue…

…nt#491)

Signed-off-by: Eduardo Silva <[email protected]>

out_td: upgrade miniz from v1.15 to > 2.0.6 (git hash b019305)

Signed-off-by: Eduardo Silva <[email protected]>

build: disable metrics on FLB_DEV mode

Signed-off-by: Eduardo Silva <[email protected]>

filter_kubernetes: if cannot merge log, only send a message on debug …

…mode

Signed-off-by: Eduardo Silva <[email protected]>

http_client: always check if the payload is set or not (fluent#352)

Signed-off-by: Eduardo Silva <[email protected]>

in_tail: fix NONBLOCK for write fd

Off by 1 error, missed setting ch_pending[1] to NONBLOCK.