Allen Institute for AI

DROP

AllenNLP, AI2 Irvine • 2019
DROP is a QA dataset that tests the comprehensive understanding of paragraphs. In this crowdsourced, adversarially-created, 96k question-answering benchmark, a system must resolve multiple references in a question, map them onto a paragraph, and perform discrete operations over them (such as addition, counting, or sorting).

Leaderboard

Top Public Submissions
Details
Created
Exact Match
1
QDGAT - ALBERT
AntGroup KG & NLP
9/8/202087%
2
Numeric Transformer - Albert
OneConnect GammaLab NYC
3/17/202086%
3
QDGAT Ensemble
AntGroup KG & NLP
12/15/201985%
4
sna_albert+ Ensemble
OneConnect GammaLab
12/3/201985%
5
QDGAT - RoBERTa
AntGroup KG & NLP
6/1/202085%

Authors

Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, Matt Gardner