Skip to content
/ logro Public
forked from templexxx/logro

Non-blocking log rolling package with page cache control in Go

License

Notifications You must be signed in to change notification settings

coinrust/logro

 
 

Repository files navigation

Logro

Deprecated

Almost all high performance log library use sync.Pool to reuse memory, so it may cause problem when just pass a pointer to logro.

Introduction

Logro is a non-blocking log rolling package with page cache control in Go. Inspired by lumberjack

Logro is built for high performance (The latency of per write is about 50 ns/op):

  • Non-blocking Write

    All write won't be blocked (Just pass a pointer then return). You will never have to worry about the log write stall impacting P999.

    When the IOPS is unusual high (may caused by bugs or unexpected behavior, e.g. 10 million/s), Logro will overwrite data on writes in lieu of blocking.

  • Write Combination

    All log data will be written to a user-space buffer first, then flush to the log file.

  • Sync in background

    Use sync in background avoiding write stall in user-facing.

    ps: OS can do the page cache flush by itself, but it may create a burst of write I/O when dirty pages hit a threshold.

  • Clean page cache

    It's meaningless to keep log files' data in page cache, so when the dirty pages are too many or we need reopen a new file, Logro will sync data to disk, then drop the page cache.

  • ...

Methods

  • WriteSyncCloser

    Logro implements such methods:

        Write(p []byte) (written int, err error)
        Sync() (err error)
        Close() (err error)
    

    Could satisfy most of log packages.

Rotation

e.g. The log file's name is a.log, the log files will be:

    a.log
    a-time.log
    a-time.log
    a-time.log
    a-time.log
    ....

Log shippers such as ELK's filebeat can set path to:

    a.log    

Control

Logro control rotation by file size only, it's simple and enough for the most cases. (Now we usually use log shippers to collect logs to databases, but not login machines and grep data)

Example

Stdlib Logger

    r, _ := New(&conf)
    log.New(r, "", log.Ldate)

Zap Logger

    r, _ := New(&conf)
    zapcore.AddSync(r)
    ...

Acknowledgments

About

Non-blocking log rolling package with page cache control in Go

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 100.0%