Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Per-database compaction daemon settings have no effect #1059

Closed
lewmelvinwork opened this issue Dec 8, 2017 · 0 comments
Closed

Per-database compaction daemon settings have no effect #1059

lewmelvinwork opened this issue Dec 8, 2017 · 0 comments
Assignees
Labels

Comments

@lewmelvinwork
Copy link

I'm using couchdb 2.0.0 on RHEL 7.2 and I'm looking to configure automatic compaction of _global_changes but I can't seem to get it to work. I've checked the file size and data size of the _global_changes database so I know the criteria I've specified have been met. I don't get an error upon couchdb startup, but nothing happens. When I tried setting a _default compaction rule, then compaction does happen for all databases including _global_changes.

Expected Behavior

Compaction of _global_changes should be triggered when the check_interval passes and the file and data sizes meet the parameters of the compaction configuration. In my testing, it doesn't get triggered.

Current Behavior

Nothing happens. There is no error in the log for the configuration, yet the check_interval passes and no compaction event occurs.

Possible Solution

I contacted the CouchDB Users email list and Adam replied with the following:

Hiya Melvin, this looks like a bug. I think what’s happening is the compaction daemon is walking the list of database shards on the node and comparing those names directly against the keys in that config block. The shard files have internal names like

shards/00000000-1fffffff/_global_changes.1512750761

If you want to test this out you could look for the full path to one of your _global_changes shards and supply that as the key instead of just “_global_changes”. Repeating the config entry for every one of the shards could also be a workaround for you until we get this patched. Can you file an issue for it at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_couchdb_issues-3F&d=DwIFaQ&c=nulvIAQnC0yOOjC0e0NVa8TOcyq9jNhjZ156R-JJU10&r=jjIKqwApUzKCDY-o1Ex3afd4bksosuJsda_NnZShUUM&m=XHjiBSrSt89_WhJ4-9-2UpcbGcs9zvxIpJN-QXPCNBI&s=dRWvHDgKxwCB7lPHOIDib7jmUy9wVGLuDcG0j0N3R7s&e=

By the way, releases prior to 1.7.1 and 2.1.1 have a fairly serious security vulnerability, it’d be good if you could upgrade. Cheers,

Adam

Steps to Reproduce (for bugs)

This is what I have in local.ini that does not work:
[compactions]
_global_changes = [{db_fragmentation, "70%"}]

Putting this into local.ini does work, but I don't want to compact all databases:
[compactions]
_default = [{db_fragmentation, "70%"}]

For the purposes of my testing, I've also added:
[compaction_daemon]
check_interval = 30

Context

_global_changes grows a lot and fills my filesystem. Enabling compaction will keep this disk growth under control.

Your Environment

  • Version used: CouchDB 2.0.0
  • Browser Name and version:
  • Operating System and version (desktop or mobile): RHEL Linux 7.2
  • Link to your project:
@kocolosk kocolosk self-assigned this Dec 14, 2017
asfgit pushed a commit that referenced this issue Dec 14, 2017
The compaction daemon was checking for custom configurations using
shard names directly, which meant that users who defined a config for
a specific DB would not see it take effect.

This patch introduces support for custom configurations for both
databases *and* shards. An example of a shard config would be

shards/00000000-1fffffff/foo.1512750761 = [{db_fragmentation, "70%"}]

We don't anticipate that to be a typical usage pattern.
@kocolosk kocolosk changed the title Automatic compaction of _global_changes doesn't work Per-database compaction daemon settings have no effect Dec 15, 2017
@kocolosk kocolosk added the bug label Dec 15, 2017
wohali pushed a commit that referenced this issue Mar 27, 2018
The compaction daemon was checking for custom configurations using
shard names directly, which meant that users who defined a config for
a specific DB would not see it take effect.

This patch introduces support for custom configurations for both
databases *and* shards. An example of a shard config would be

shards/00000000-1fffffff/foo.1512750761 = [{db_fragmentation, "70%"}]

We don't anticipate that to be a typical usage pattern.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants