Quoref

AllenNLP, AI2 Irvine • 2019

Quoref is a QA dataset which tests the coreferential reasoning capability of reading comprehension systems. In this span-selection benchmark containing 24K questions over 4.7K paragraphs from Wikipedia, a system must resolve hard coreferences before selecting the appropriate span(s) in the paragraphs for answering questions.

Download Read Paper View Website

License: CC BY

Leaderboard

Top Public Submissions

Details	Created	Exact Match
1 TASE - CorefRoBERTa Tsinghua University (Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Maosong Sun, Zhiyuan Liu)	5/15/2020	81%
2 TASE-CoNLL-joint-qgen Mingzhu Wu, Nafise Sadat Moosavi, Dan Roth, Iryna Gurevych	12/14/2020	80%
3 TASE - RoBERTa Elad Segal, Avia Efrat, Mor Shoham, Amir Globerson, Jonathan Berant	12/8/2019	80%
4 CorefRoberta-Large Tsinghua University (Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Maosong Sun, Zhiyuan Liu)	3/25/2020	76%
5 RoBERTa-MT WeChatAI	12/9/2019	73%

View Leaderboard

Authors

Pradeep Dasigi, Nelson F. Liu, Ana Marasović, Noah A. Smith, Matt Gardner