Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement function to convert legacy filters to new style #6314

Merged
merged 4 commits into from
Nov 20, 2023

Conversation

silvanocerza
Copy link
Contributor

Related Issues

Proposed Changes:

This PR introduces an utility function to ease development of new Document Stores for Haystack 2.x.

The function is meant to be used to convert legacy style filters to the newly proposed specification from #6001. This way Document Stores developers will only need to implement logic for the new specification.

This will also make it easier for user to migrate from Haystack 1.x to 2.x.

How did you test it?

I added new tests for the convert() function.

Notes for the reviewer

N/A

Checklist

@silvanocerza silvanocerza added the 2.x Related to Haystack v2.0 label Nov 15, 2023
@silvanocerza silvanocerza self-assigned this Nov 15, 2023
@silvanocerza silvanocerza requested review from a team as code owners November 15, 2023 17:00
@silvanocerza silvanocerza requested review from ZanSara and removed request for a team November 15, 2023 17:00
@github-actions github-actions bot added topic:tests type:documentation Improvements on the docs labels Nov 15, 2023
Copy link
Contributor

@agnieszka-m agnieszka-m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just minor typos.

haystack/preview/utils/filters.py Outdated Show resolved Hide resolved
releasenotes/notes/filters-converter-485cd24cf38407d0.yaml Outdated Show resolved Hide resolved
Co-authored-by: Agnieszka Marzec <[email protected]>
Copy link
Contributor

@ZanSara ZanSara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! 🚀

```
"""
converted = _internal_convert(filters)
if "conditions" not in converted:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure this happens only in case of the simple filters you describe below? I would have handled this in the _internal_convert function (or somewhere else where it makes sense) and leave this check for conditions that really didn't get converter, to raise an exception.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try handling it in _internal_convert() so the logic would have been contained in a single function, but that complicated the logic quite a bit.

The main issue is that something similar to this is actually already handled in _internal_convert(), at line 379 to be precise. But it only runs when you have at least 2 conditions.

I tried handling an implicit root $and with a single condition but it overcomplicates the logic for close to no gain. Also it makes the logic harder to understand.

This solution still handles it pretty gracefully so I'm hapy with it.

Also considering that it's only meant for legacy filters, hopefully in the future everyone will use the new filters.

@silvanocerza silvanocerza merged commit 83c245d into main Nov 20, 2023
22 checks passed
@silvanocerza silvanocerza deleted the filters-converter branch November 20, 2023 12:00
vblagoje pushed a commit that referenced this pull request Nov 22, 2023
* Implement function to convert legacy filters to new style

* Reduce return statements in conversion to fix linting

* Move convert function in different module

* Fix typos in docstrings

Co-authored-by: Agnieszka Marzec <[email protected]>

---------

Co-authored-by: Agnieszka Marzec <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add filtering logic to query DocumentStores
3 participants