Skip to content

No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval

Notifications You must be signed in to change notification settings

guilhermemr04/scaling-zero-shot-retrieval

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval

This repository contains the code to reproduce the results presented in the paper No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval.

In this work, we show that increasing model size results in marginal gains on in-domain test sets, but much larger gains in new domains never seen during fine-tuning. Furthermore, we show that rerankers largely outperform dense ones of similar size in several tasks. Our largest reranker reaches the state of the art in 12 of the 18 datasets of the Benchmark-IR (BEIR).

Ilustration of our results

Models

How do I reproduce the results?

To reproduce monoT5-3B results at least 25GB of RAM and a Tesla P100 GPU are required.

How do I cite this work?

 @article{Rosa_2022,
    title={No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval},
    author={Rosa, Guilherme and Bonifacio, Luiz and Jeronymo, Vitor and Abonizio, Hugo and Fadaee, Marzieh and Lotufo, Roberto and Nogueira, Rodrigo},
    journal={https://arxiv.org/abs/2206.02873},
    year={2022}
}

About

No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published