Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

copytruncate not empty the file #486

Open
ixiaoyi93 opened this issue Jan 5, 2023 · 7 comments
Open

copytruncate not empty the file #486

ixiaoyi93 opened this issue Jan 5, 2023 · 7 comments

Comments

@ixiaoyi93
Copy link

Since the application is not aware of file descriptor changes, copytruncate is used. for example, cutting the ssf-json.log file，the following problems were encountered in the process of using it：

  • The cut log file is not emptied and the file grows larger over time.
$ ls -alh /data/logs/dev/user/tomcat-user
total 1.4G
drwsr-sr-t 2 4294967294 4294967294 4.0K Jan  5 14:20 .
drwxr-xr-x 4 4294967294 4294967294 4.0K Sep 15 17:39 ..
-rw-r--r-- 1 4294967294 4294967294 155M Jan  5 14:32 ssf-console.log
-rw-r--r-- 1 4294967294 4294967294 150M Jan  5 11:30 ssf-console.log-2023-01-04
-rw-r--r-- 1 4294967294 4294967294  19M Jan  5 14:32 ssf-error.log
-rw-r--r-- 1 4294967294 4294967294  18M Jan  5 11:30 ssf-error.log-2023-01-04
-rw-r--r-- 1 4294967294 4294967294 553M Jan  5 14:31 ssf-json.log
-rw-r--r-- 1 4294967294 4294967294 539M Jan  5 11:31 ssf-json.log-2023-01-04
  • Another problem is that when using commands such as vim, less, etc., there is no response for a long time, and the file is very slow to open.
    image

The logrotate version is 3.21.0, configuration file is as follows.

$ cat /etc/logrotate.d/appLogs
/data/logs/dev/**/**/ssf-console.log
/data/logs/dev/**/**/ssf-error.log
/data/logs/dev/**/**/ssf-json.log
/data/logs/test/**/**/ssf-console.log
/data/logs/test/**/**/ssf-error.log
/data/logs/test/**/**/ssf-json.log
{
        hourly
        missingok
        rotate 1
        copytruncate
        nocompress
        delaycompress
        ifempty
        dateext
        dateformat -%Y-%m-%d
        dateyesterday
}
@cgzones
Copy link
Member

cgzones commented Jan 5, 2023

Did logrotate issue any errors on rotation (journalctl -u logrotate.service)?

@ixiaoyi93
Copy link
Author

ixiaoyi93 commented Jan 6, 2023 •
edited

@cgzones I am using the command /usr/local/logrotate-3.21.0/logrotate -f /etc/logrotate.d/appLogs, not using systemd management logrotate.service.
Logfiles are rotated properly, but the rotated files are not cleared.

@kdudka
Copy link
Member

kdudka commented Jan 6, 2023

@xiaomuyi Could you please run the command with -v and paste its output here?

@ixiaoyi93
Copy link
Author

@xiaomuyi Could you please run the command with -v and paste its output here?

@kdudka Command execution and console output.

$ /usr/local/logrotate-3.21.0/logrotate -v -f /etc/logrotate.conf
rotating pattern: /data/logs/dev/upload/alpha-upload/ssf-console.log
/data/logs/dev/upload/alpha-upload/ssf-error.log
/data/logs/dev/upload/alpha-upload/ssf-json.log
 forced from command line (15 rotations)
empty log files are not rotated, old logs are removed
considering log /data/logs/dev/upload/alpha-upload/ssf-console.log
  Now: 2023-01-09 13:44
  Last rotated at 2023-01-09 13:37
  log needs rotating
considering log /data/logs/dev/upload/alpha-upload/ssf-error.log
  log /data/logs/dev/upload/alpha-upload/ssf-error.log does not exist -- skipping
considering log /data/logs/dev/upload/alpha-upload/ssf-json.log
  Now: 2023-01-09 13:44
  Last rotated at 2023-01-09 13:37
  log needs rotating
rotating log /data/logs/dev/upload/alpha-upload/ssf-console.log, log->rotateCount is 15
Converted '-%Y-%m-%d' -> '-%Y-%m-%d'
dateext suffix '-2023-01-08'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
glob finding old rotated logs failed
copying /data/logs/dev/upload/alpha-upload/ssf-console.log to /data/logs/dev/upload/alpha-upload/ssf-console.log-2023-01-08
truncating /data/logs/dev/upload/alpha-upload/ssf-console.log
rotating log /data/logs/dev/upload/alpha-upload/ssf-json.log, log->rotateCount is 15
Converted '-%Y-%m-%d' -> '-%Y-%m-%d'
dateext suffix '-2023-01-08'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
glob finding old rotated logs failed
copying /data/logs/dev/upload/alpha-upload/ssf-json.log to /data/logs/dev/upload/alpha-upload/ssf-json.log-2023-01-08
truncating /data/logs/dev/upload/alpha-upload/ssf-json.log

rotating pattern: /data/logs/dev/user/tomcat-user/ssf-console.log
/data/logs/dev/user/tomcat-user/ssf-error.log
/data/logs/dev/user/tomcat-user/ssf-json.log
 forced from command line (15 rotations)
empty log files are not rotated, old logs are removed
considering log /data/logs/dev/user/tomcat-user/ssf-console.log
  Now: 2023-01-09 13:44
  Last rotated at 2023-01-09 13:37
  log needs rotating
considering log /data/logs/dev/user/tomcat-user/ssf-error.log
  Now: 2023-01-09 13:44
  Last rotated at 2023-01-09 13:37
  log needs rotating
considering log /data/logs/dev/user/tomcat-user/ssf-json.log
  Now: 2023-01-09 13:44
  Last rotated at 2023-01-09 13:37
  log needs rotating
rotating log /data/logs/dev/user/tomcat-user/ssf-console.log, log->rotateCount is 15
Converted '-%Y-%m-%d' -> '-%Y-%m-%d'
dateext suffix '-2023-01-08'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
glob finding old rotated logs failed
copying /data/logs/dev/user/tomcat-user/ssf-console.log to /data/logs/dev/user/tomcat-user/ssf-console.log-2023-01-08
truncating /data/logs/dev/user/tomcat-user/ssf-console.log
rotating log /data/logs/dev/user/tomcat-user/ssf-error.log, log->rotateCount is 15
Converted '-%Y-%m-%d' -> '-%Y-%m-%d'
dateext suffix '-2023-01-08'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
glob finding old rotated logs failed
copying /data/logs/dev/user/tomcat-user/ssf-error.log to /data/logs/dev/user/tomcat-user/ssf-error.log-2023-01-08
truncating /data/logs/dev/user/tomcat-user/ssf-error.log
rotating log /data/logs/dev/user/tomcat-user/ssf-json.log, log->rotateCount is 15
Converted '-%Y-%m-%d' -> '-%Y-%m-%d'
dateext suffix '-2023-01-08'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
glob finding old rotated logs failed
copying /data/logs/dev/user/tomcat-user/ssf-json.log to /data/logs/dev/user/tomcat-user/ssf-json.log-2023-01-08
truncating /data/logs/dev/user/tomcat-user/ssf-json.log

logrotate.conf configuration file

$  cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
# weekly

# keep 4 weeks worth of backlogs
# rotate 4

# create new (empty) log files after rotating old ones
# create

# use date as a suffix of the rotated file
# dateext

# uncomment this if you want your log files compressed
# compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

/etc/logrotate.d part of the configuration file in the directory，

$ cat /etc/logrotate.d/upload
/data/logs/dev/upload/alpha-upload/ssf-console.log
/data/logs/dev/upload/alpha-upload/ssf-error.log
/data/logs/dev/upload/alpha-upload/ssf-json.log
{
            daily
            rotate 15
            copytruncate
            nocompress
            notifempty
            missingok
            dateext
            dateformat -%Y-%m-%d
            dateyesterday
}

$ cat /etc/logrotate.d/user
/data/logs/dev/user/tomcat-user/ssf-console.log
/data/logs/dev/user/tomcat-user/ssf-error.log
/data/logs/dev/user/tomcat-user/ssf-json.log
{
            daily
            rotate 15
            copytruncate
            nocompress
            notifempty
            missingok
            dateext
            dateformat -%Y-%m-%d
            dateyesterday
}

The result of log file cutting is that some of the files can be cut properly and the situation. But then again some of the logs are cut but the cut files are not cleared. I don't know why this is happening? Is it because there are too many configuration files?

$ ls -alh /data/logs/dev/user/tomcat-user/
total 319M
drwsr-sr-t 2 4294967294 4294967294 4.0K Jan  9 13:44 .
drwxr-xr-x 4 4294967294 4294967294 4.0K Sep 15 17:39 ..
-rw-r--r-- 1 4294967294 4294967294    0 Jan  9 13:44 ssf-console.log
-rw-r--r-- 1 4294967294 4294967294  13M Jan  6 00:00 ssf-console.log-2023-01-05-2023010600.backup
-rw-r--r-- 1 4294967294 4294967294  97M Jan  9 13:44 ssf-console.log-2023-01-08
-rw-r--r-- 1 4294967294 4294967294    0 Jan  9 13:44 ssf-error.log
-rw-r--r-- 1 4294967294 4294967294 7.1M Jan  9 13:44 ssf-error.log-2023-01-08
-rw-r--r-- 1 4294967294 4294967294    0 Jan  9 13:44 ssf-json.log
-rw-r--r-- 1 4294967294 4294967294 202M Jan  9 13:44 ssf-json.log-2023-01-08
$ ls -alh /data/logs/dev/upload/alpha-upload
total 21M
drwsr-sr-t 2 4294967294 4294967294 4.0K Jan  9 13:44 .
drwxr-xr-x 3 4294967294 4294967294 4.0K Sep 15 17:40 ..
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  2 00:00 access_log.2023-01-01.log
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  3 00:00 access_log.2023-01-02.log
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  4 00:00 access_log.2023-01-03.log
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  5 00:00 access_log.2023-01-04.log
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  6 00:00 access_log.2023-01-05.log
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  7 00:00 access_log.2023-01-06.log
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  8 00:00 access_log.2023-01-07.log
-rw-r--r-- 1 4294967294 4294967294 2.4M Jan  9 00:00 access_log.2023-01-08.log
-rw-r--r-- 1 4294967294 4294967294 1.4M Jan  9 13:44 access_log.2023-01-09.log
-rw-r--r-- 1 4294967294 4294967294 119K Jan  9 13:44 ssf-console.log
-rw-r--r-- 1 4294967294 4294967294 118K Jan  9 13:44 ssf-console.log-2023-01-08
-rw-r--r-- 1 4294967294 4294967294 171K Jan  9 13:44 ssf-json.log
-rw-r--r-- 1 4294967294 4294967294  10K Jan  6 00:00 ssf-json.log-2023-01-05-2023010600.backup
-rw-r--r-- 1 4294967294 4294967294 171K Jan  9 13:44 ssf-json.log-2023-01-08

@kdudka
Copy link
Member

kdudka commented Jan 9, 2023

Could you please paste the output of du /data/logs/dev/upload/alpha-upload/ssf-*?

@ixiaoyi93
Copy link
Author

ixiaoyi93 commented Jan 9, 2023 •
edited

Could you please paste the output of du /data/logs/dev/upload/alpha-upload/ssf-*?

$ du /data/logs/dev/upload/alpha-upload/ssf-*
8	/data/logs/dev/upload/alpha-upload/ssf-console.log
4	/data/logs/dev/upload/alpha-upload/ssf-console.log-2023-01-08
12	/data/logs/dev/upload/alpha-upload/ssf-json.log
12	/data/logs/dev/upload/alpha-upload/ssf-json.log-2023-01-05-2023010600.backup
4	/data/logs/dev/upload/alpha-upload/ssf-json.log-2023-01-08

@kdudka
Copy link
Member

kdudka commented Jan 9, 2023

There is no error in logrotate's output. I think the files are truncated properly but the service that creates the log files then continues writing to them at the original offset, which in turn restores the apparent size of the files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kdudka @cgzones @ixiaoyi93