Skip deletes / tombstone eviction

[wohali]

@janl:

ddoc filter to specify docs to drop during compaction

This one is self-explanatory, mobile clients usually don’t need to sync deletes from a year ago first. Mango filters might already get us there, maybe we can do better.

@davisp:

There's two issues we'll need to consider. First, my memory is a bit hazy but if I recall correctly there's a subtle issue around modifying the contents of a database during compaction. I believe it has to do with resuming compaction but I'd have to spend more time trying to write this again.

The second issue is that in a cluster if any of the documents that are dropped are accessed its likely that read-repair will end up rewriting them to the database as from the clustered point of view if a doc is missing on a single node then it needs to be re-written to heal the error.

[janl]

This needs review in light of the recent clustered purge feature, since that had the same issue to work with

[bdoyle0182]

Is this reasonable to implement in the 3.x series if clustered purge had to deal with the addressed issues?

[jjrodrig]

My use cases are mainly in a mobile context where mobile clients don't need the old deleted docs. Currently, we are cleaning user databases writing external logic using the clustered purge functionality. This process needs to query the database to identify the candidate doc revisions to be purged, and then purge the leaf revisions in batches which is a quite expensive process.

Having this logic defined as part of the database makes the system simpler as CouchDB can manage it without the need for extra code as part of internal maintenance processes. So, I find this a good functionality to keep in mind as part of CouchDB.