Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to weaviate-client to v4 #40194

Merged
merged 52 commits into from
Jun 26, 2024
Merged

Conversation

Lee-W
Copy link
Member

@Lee-W Lee-W commented Jun 12, 2024

Why

According to Weaviate Python client (v4) goes GA, the v3 client is now deprecated and we need to upgrade to v4

What

  • update min weaviate-client version to 4.4.0
  • weaviate hook
    • update airflow connection to v4 style
    • migrate the following methods to v4 API
      • test_connections
      • create_class to create_collection
      • get_schema to get_collection_configuraiton
      • delete_classes to delete_collections
      • query_with_vector
      • query_without_vector to query_with_text
      • create_object
      • get_object
      • delete_object
      • update_object
      • replace_object
      • object_exists
      • batch_data
      • get_or_create_object
      • update_collection_configuration
      • _generate_uuids
      • create_or_replace_document_objects
      • _delete_objects
      • _delete_all_documents_objects
      • _get_documents_to_uuid_map
      • _get_segregated_documents
    • remove the following methods which are no longer supported in v4 API
      • validate_object
      • update_schema
      • create_schema
      • delete_all_schema
      • create_or_replace_classes
      • _compare_schema_subset
      • _convert_properties_to_dict
      • check_subset_of_schema
    • remove deprecated method get_client
    • remove unused argument retry_status_codes in __init__
  • weaviate operators
    • migrate WeaviateDocumentIngestOperator and WeaviateIngestOperator to use hook with v4 API

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@Lee-W Lee-W force-pushed the upgrade-to-weaviate-v4 branch 10 times, most recently from 12f8cca to 2b594c8 Compare June 20, 2024 06:47
@Lee-W Lee-W changed the title Upgrade to weaviate v4 Upgrade to weaviate-client to v4 Jun 20, 2024
@Lee-W Lee-W marked this pull request as ready for review June 20, 2024 08:26
@Lee-W
Copy link
Member Author

Lee-W commented Jun 20, 2024

Hi @eladkal , if my memory serves me right, when we make a PR with breaking changes to a provider, we need to mark it somewhere. Could you please point out where I should mark it? Thanks!

@potiuk
Copy link
Member

potiuk commented Jun 20, 2024

Hi @eladkal , if my memory serves me right, when we make a PR with breaking changes to a provider, we need to mark it somewhere. Could you please point out where I should mark it? Thanks!

In provider's CHANGELOG.rst (see comment at the top of it).

But is it really a breaking change for the users of the Hook/Operator?

@potiuk
Copy link
Member

potiuk commented Jun 20, 2024

Looked at it - there are many changes and it does look like breaking looking at the few of those. In CHANGELOG you should describe what the users of previous version should do to migrate to this one.

Lee-W added 27 commits June 25, 2024 18:34
…perties_to_dict, check_subset_of_schema as there's no v4 API counterpart
@Lee-W Lee-W merged commit dcaf82a into apache:main Jun 26, 2024
108 checks passed
@Lee-W Lee-W deleted the upgrade-to-weaviate-v4 branch June 26, 2024 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants