DROP

AllenNLP, AI2 Irvine • 2019

DROP is a QA dataset that tests the comprehensive understanding of paragraphs. In this crowdsourced, adversarially-created, 96k question-answering benchmark, a system must resolve multiple references in a question, map them onto a paragraph, and perform discrete operations over them (such as addition, counting, or sorting).

Download Read Paper View Website View Repo

License: CC BY

Leaderboard

Top Public Submissions

Details	Created	Exact Match
1 QDGAT - ALBERT AntGroup KG & NLP	9/8/2020	87%
2 Numeric Transformer - Albert OneConnect GammaLab NYC	3/17/2020	86%
3 QDGAT Ensemble AntGroup KG & NLP	12/15/2019	85%
4 sna_albert+ Ensemble OneConnect GammaLab	12/3/2019	85%
5 QDGAT - RoBERTa AntGroup KG & NLP	6/1/2020	85%

View Leaderboard

Authors

Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, Matt Gardner