Quoref
AllenNLP, AI2 Irvine • 2019
Quoref is a QA dataset which tests the coreferential reasoning capability of reading comprehension systems. In this span-selection benchmark containing 24K questions over 4.7K paragraphs from Wikipedia, a system must resolve hard coreferences before selecting the appropriate span(s) in the paragraphs for answering questions.
Leaderboard
Top Public Submissions
Details | Created | Exact Match |
|---|---|---|
1 TASE - CorefRoBERTa Tsinghua University (Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Maosong Sun, Zhiyuan Liu) | 5/15/2020 | 81% |
2 TASE - RoBERTa Elad Segal*, Avia Efrat*, Mor Shoham*, Amir Globerson, Jonathan Berant | 12/8/2019 | 80% |
3 CorefRoberta-Large Tsinghua University (Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Maosong Sun, Zhiyuan Liu) | 3/25/2020 | 76% |
4 RoBERTa-MT WeChatAI | 12/9/2019 | 73% |
5 XLNet Baseline AllenNLP (Dasigi, Liu, Marasović, Smith, and Gardner) | 9/6/2019 | 62% |
Authors
Pradeep Dasigi, Nelson F. Liu, Ana Marasović, Noah A. Smith, Matt Gardner