Tokenizer warnings in Dense Passage retriever update embeddings in Elasticsearch (ref tutorial 6)

[usaraj]

Environment
Using the latest stable git branch.
Colab
Refer Tutorial 6

Question
Do we need to pass in additional arguments either to the retriever call or to the document_store.update_embeddings call to avoid these warning from transformers.tokenization...?

While using the Dense Passage retriever to store the embeddings in Elasticsearch using the command :

document_store.update_embeddings(retriever)

there is a warning as pasted in additional context section.

Additional context
Output of Warning:

07/15/2020 23:42:29 - WARNING - transformers.tokenization_utils_base - Truncation was not explicitely activated but max_length is provided a specific value, please use truncation=True to explicitely truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to truncation.

[tholor]

Hey @usaraj ,

Thanks for reporting this. The warning was introduced with the update to the latest transformers version (3.0.2). While this doesn't impact any functionality, I refactored the tokenization in DPR to avoid the warning (see #239 ).