Skip to content

Commit

Permalink
Log pending compaction bytes in a couple places (facebook#12267)
Browse files Browse the repository at this point in the history
Summary:
This PR adds estimated pending compaction bytes in two places:

- The "Level summary", which is printed to the info LOG after every flush or compaction
- The "rocksdb.cfstats" property, which is printed to the info LOG periodically according to `stats_dump_period_sec`

Pull Request resolved: facebook#12267

Test Plan:
Ran `./db_bench -benchmarks=filluniquerandom -stats_dump_period_sec=1 -statistics=true -write_buffer_size=524288` and looked at the LOG.

```
** Compaction Stats [default] **
...
Estimated pending compaction bytes: 12117691
...
2024/01/22-13:15:12.283563 1572872 (Original Log Time 2024/01/22-13:15:12.283540) [/db_impl/db_impl_compaction_flush.cc:371] [default] Level summary: files[10 1 0 0 0 0 0] max score 0.50, estimated pending compaction bytes 12359137
```

Reviewed By: cbi42

Differential Revision: D52973337

Pulled By: ajkr

fbshipit-source-id: c4e546bd9bdac387eebeeba303d04125212037b8
  • Loading branch information
ajkr authored and facebook-github-bot committed Jan 23, 2024
1 parent 84711e2 commit 7fe9316
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
6 changes: 6 additions & 0 deletions db/internal_stats.cc
Original file line number Diff line number Diff line change
Expand Up @@ -2067,6 +2067,12 @@ void InternalStats::DumpCFStatsNoFileHistogram(bool is_periodic,
interval_compact_bytes_read / kMB / std::max(interval_seconds_up, 0.001),
interval_compact_micros / kMicrosInSec);
value->append(buf);

snprintf(buf, sizeof(buf),
"Estimated pending compaction bytes: %" PRIu64 "\n",
vstorage->estimated_compaction_needed_bytes());
value->append(buf);

if (is_periodic) {
cf_stats_snapshot_.compact_bytes_write = compact_bytes_write;
cf_stats_snapshot_.compact_bytes_read = compact_bytes_read;
Expand Down
6 changes: 4 additions & 2 deletions db/version_set.cc
Original file line number Diff line number Diff line change
Expand Up @@ -4546,8 +4546,10 @@ const char* VersionStorageInfo::LevelSummary(
// overwrite the last space
--len;
}
len += snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len,
"] max score %.2f", compaction_score_[0]);
len +=
snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len,
"] max score %.2f, estimated pending compaction bytes %" PRIu64,
compaction_score_[0], estimated_compaction_needed_bytes_);

if (!files_marked_for_compaction_.empty()) {
snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len,
Expand Down

0 comments on commit 7fe9316

Please sign in to comment.