DROP
AllenNLP, AI2 Irvine • 2019
DROP is a QA dataset that tests the comprehensive understanding of paragraphs. In this crowdsourced, adversarially-created, 96k question-answering benchmark, a system must resolve multiple references in a question, map them onto a paragraph, and perform discrete operations over them (such as addition, counting, or sorting).
Leaderboard
Top Public Submissions
Details | Created | Exact Match |
|---|---|---|
1 Numeric Transformer - Albert OneConnect GammaLab NYC | 3/17/2020 | 86% |
| 12/15/2019 | 85% | |
3 sna_albert+ Ensemble OneConnect GammaLab | 12/3/2019 | 85% |
| 6/1/2020 | 85% | |
5 Numeric Transformer - RoBERTa OneConnect GammaLab NYC | 3/3/2020 | 84% |
Authors
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, Matt Gardner