Improve performance for updating & querying vector_ids in the FAISSDocumentStore

[tanaysoni]

The vector_id from FAISS gets stored as a metadata field in the FAISSDocumentStore.

When mebeddings are updated, the FAISSDocumentStore.update_embeddings() method calls SQLDocumentStore.update_document_meta() to replace the entire document metadata with the new vector_id. This takes a considerable time as all metadata fields get updated.

A possible solution is to selectively update only the vector_id meta field from within the FAISSDocumentStore. Having the updates within the FAISSDocumentStore would also enable to commit SQL transactions in a batch rather than for each document.

[tholor]

Let's prioritize this one. In our new benchmarks, we found that the current SQL schema is also slowing down queries with FAISSDocumentStore significantly:

• 100 queries on 50k DPR embeddings with HNSWFlat, InnerProduct
• with SQL: 0.284 sec / query
• without SQL: 0.0186 sec / query

So this is quite a blocker for large scale usage of FAISS and appropriate benchmarking.