Update LoggingHandler to support logging per interval

[liuzh47]

Description

Add support of logging per interval to the default LoggingHandler. It acts as a middle ground between logging per batch and logging per epoch. Please refer to the description in Issue

Fixes #16918

[eric-haibin-lin]

Shall we add a unit test for this?

[liuzh47]

Shall we add a unit test for this?

What assertion shall we make in this case?

[leezu]

You could assert the number of lines logged. For example, if you specify training for 10 batches and log every 5 batches, you may test for two lines being logged.

[roywei]

Thanks for the fix. Shall we remove LOG_PER_BATCH as it can be replaced by LOG_PER_INTERVAL?
Since this is in contrib, I think it's fine and less options will be easier for users.

[liuzh47]

You could assert the number of lines logged. For example, if you specify training for 10 batches and log every 5 batches, you may test for two lines being logged.

Ok, I'll take a try.

[liuzh47]

Thanks for the fix. Shall we remove LOG_PER_BATCH as it can be replaced by LOG_PER_INTERVAL?
Since this is in contrib, I think it's fine and less options will be easier for users.

Thanks for the suggestion.

I have thought about the option before. Most parts of logging batch and logging interval are overlapped. But during batch logging, we only logged the training metrics:

            for metric in self.train_metrics:
                # only log current training loss & metric after each batch
                name, value = metric.get()
                msg += '%s: %.4f, ' % (name, value)

But during interval logging, I think it is better to log both training and validation metrics. So LOG_PER_BATCH is still necessary?

          for monitor in self.train_metrics + self.val_metrics:
                name, value = monitor.get()
                msg += '%s: %.4f, ' % (name, value)

Or we can merge LOG_PER_BATCH and LOG_PER_INTERVAL, then we use self.log_interval to check whether using validation metrics or not.

[leezu]

I'm actually not sure if logging both self.train_metrics + self.val_metrics during LOG_PER_INTERVAL is good.
Currently the notion of val_metrics and train_metrics is not clearly decoupled. Thinking about how to tackle #16959 may also help to clarify the relation of eval_metrics and train_metrics and how their values should be updated and logged.

[liuzh47]

I'm actually not sure if logging both self.train_metrics + self.val_metrics during LOG_PER_INTERVAL is good.
Currently the notion of val_metrics and train_metrics is not clearly decoupled. Thinking about how to tackle #16959 may also help to clarify the relation of eval_metrics and train_metrics and how their values should be updated and logged.

It makes sense to me. I'll merge LOG_PER_BATCH and LOG_PER_INTERVAL and leave out the val_metrics.