SciFact

Semantic Scholar • 2020

Due to the rapid growth in the scientific literature, there is a need for automated systems to assist researchers and the public in assessing the veracity of scientific claims. To facilitate the development of systems for this task, we introduce SciFact, a dataset of 1.4K expert-written claims, paired with evidence-containing abstracts annotated with veracity labels and rationales.

Download Read Paper View Website View Repo

License: CC BY-NC 2.0

Leaderboard

Top Public Submissions

Details	Created	Abstract Label-Only (R)
1 VerT5erini (2-stage Neural Retrieval) (Train+Dev) University of Waterloo	2/19/2021	73%
2 ParagraphJoint PLUS Lab: Xiangci Li (UT Dallas), Gully Burns (Chan Zuckerburg Initiative), Nanyun Peng (UCLA)	1/26/2021	64%
3 VerT5erini (2-stage Neural Retrieval) University of Waterloo	1/26/2021	66%
4 Law & Econ fine-tuning the e-FEVER sytem (https://truthandtrustonline.com/wp-content/uploads/2020/10/TTO04.pdf) on the scifact dataset + some tricks for improved evidence retrieval. more details, an academic paper and a link to a github repository with the code will follow shortly	2/11/2021	63%
5 VerT5erini (BM25 Retrieval) University of Waterloo	1/27/2021	61%

View Leaderboard

Authors

David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi