Allen Institute for AI

DROP

AllenNLP, AI2 Irvine • 2019
DROP is a QA dataset that tests the comprehensive understanding of paragraphs. In this crowdsourced, adversarially-created, 96k question-answering benchmark, a system must resolve multiple references in a question, map them onto a paragraph, and perform discrete operations over them (such as addition, counting, or sorting).

Leaderboard

Top Public Submissions
Details
Created
Exact Match
1
Numeric Transformer - Albert
OneConnect GammaLab NYC
3/17/202086%
12/15/201985%
3
sna_albert+ Ensemble
OneConnect GammaLab
12/3/201985%
6/1/202085%
5
Numeric Transformer - RoBERTa
OneConnect GammaLab NYC
3/3/202084%

Authors

Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, Matt Gardner