Rankt5: Fine-tuning t5 for text ranking with ranking losses
Proceedings of the 46th International ACM SIGIR Conference on Research and …, 2023•dl.acm.org
Pretrained language models such as BERT have been shown to be exceptionally effective
for text ranking. However, there are limited studies on how to leverage more powerful
sequence-to-sequence models such as T5. Existing attempts usually formulate text ranking
as a classification problem and rely on postprocessing to obtain a ranked list. In this paper,
we propose RankT5 and study two T5-based ranking model structures, an encoder-decoder
and an encoder-only one, so that they not only can directly output ranking scores for each …
for text ranking. However, there are limited studies on how to leverage more powerful
sequence-to-sequence models such as T5. Existing attempts usually formulate text ranking
as a classification problem and rely on postprocessing to obtain a ranked list. In this paper,
we propose RankT5 and study two T5-based ranking model structures, an encoder-decoder
and an encoder-only one, so that they not only can directly output ranking scores for each …
Pretrained language models such as BERT have been shown to be exceptionally effective for text ranking. However, there are limited studies on how to leverage more powerful sequence-to-sequence models such as T5. Existing attempts usually formulate text ranking as a classification problem and rely on postprocessing to obtain a ranked list. In this paper, we propose RankT5 and study two T5-based ranking model structures, an encoder-decoder and an encoder-only one, so that they not only can directly output ranking scores for each query-document pair, but also can be fine-tuned with pairwise or listwise ranking losses to optimize ranking performance. Our experiments show that the proposed models with ranking losses can achieve substantial ranking performance gains on different public text ranking data sets. Moreover, ranking models fine-tuned with listwise ranking losses have better zero-shot ranking performance on out-of-domain data than models fine-tuned with classification losses.