DROP
AllenNLP, AI2 Irvine • 2019
DROP is a QA dataset that tests the comprehensive understanding of paragraphs. In this crowdsourced, adversarially-created, 96k question-answering benchmark, a system must resolve multiple references in a question, map them onto a paragraph, and perform discrete operations over them (such as addition, counting, or sorting).
Leaderboard
Top Public Submissions
Details | Created | Exact Match |
|---|---|---|
1 QDGAT - ALBERT AntGroup KG & NLP | 9/8/2020 | 87% |
2 Numeric Transformer - Albert OneConnect GammaLab NYC | 3/17/2020 | 86% |
3 QDGAT Ensemble AntGroup KG & NLP | 12/15/2019 | 85% |
4 sna_albert+ Ensemble OneConnect GammaLab | 12/3/2019 | 85% |
5 QDGAT - RoBERTa AntGroup KG & NLP | 6/1/2020 | 85% |
Authors
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, Matt Gardner