Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8190 optimized filter by id #8191

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

jeffsenn
Copy link

Copied from issue description.

Issue

A replicate.from feed with a client side filter from a large/active remote when you just want to keep existing (already fetched) documents up-to-date is extremely inefficient.

Info

When you specify filter:a_function on a replicate, PouchDB forces the changes feed (inside the replication) to specify include_docs:true. This will result in all remote changes coming over the wire. Normally this is what you want, however in the (extremely interesting) case of only wanting to keep documents you have already fetched up-to-date, it is very inefficient -- all you really need is the document _id.

With 2 small changes this can be made much more efficient.

One user visible API change: support a filter_id_only: true option to replicate. If specified this suppresses the additional include_docs:true and sends the change to the filter function as {_id: doc_id}

I will followup with a PR in case you want to include this feature

@jeffsenn
Copy link
Author

CI Failures do not look like they are due to this PR.

@janl
Copy link
Contributor

janl commented Feb 12, 2021

why would you use a filter instead of the doc_ids option for replication?

@dheimoz
Copy link

dheimoz commented Oct 19, 2021

hello @jeffsenn Would you like to have this issue reviewed? @janl had a doubt about the purpose.
As you have seen, tests have been migrated to Github Actions.

@lucidNTR
Copy link
Contributor

@janl i think i understand the intend. with the doc_ids option it is only possible to filter for up front known doc ids. what @jeffsenn wants to do is check live in the filter function if an incoming doc id from the feed is an id that is in the local pouch without getting alls docs bodies. for dbs where there are many changes and large docs of which you just want a small subset locally, this could make quite a difference. is there any other way to accomplish this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants