Papers
See AI2's Award Winning Papers
Learn more about AI2's Lasting Impact Award
Viewing 121-130 of 155 papers
SciREX: A Challenge Dataset for Document-Level Information Extraction
Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, Iz BeltagyACL • 2020 Extracting information from full documents is an important problem in many domains, but most previous work focus on identifying relationships within a sentence or a paragraph. It is challenging to create a large-scale information extraction (IE) dataset at…SPECTER: Document-level Representation Learning using Citation-informed Transformers
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. WeldACL • 2020 Representation learning is a critical ingredient for natural language processing systems. Recent Transformer language models like BERT learn powerful textual representations, but these models are targeted towards tokenand sentence-level training objectives…Stolen Probability: A Structural Weakness of Neural Language Models
David Demeter, Gregory Kimmel, Doug DowneyACL • 2020 Neural Network Language Models (NNLMs) generate probability distributions by applying a softmax function to a distance metric formed by taking the dot product of a prediction vector with all word vectors in a high-dimensional embedding space. The dot-product…TREC-COVID: Constructing a Pandemic Information Retrieval Test Collection
Ellen M. Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R. Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, Lucy Lu Wang arXiv • 2020 TREC-COVID is a community evaluation designed to build a test collection that captures the information needs of biomedical researchers using the scientific literature during a pandemic. One of the key characteristics of pandemic search is the accelerated rate…TREC-COVID: Rationale and Structure of an Information Retrieval Shared Task for COVID-19
Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen M. Voorhees, Lucy Lu Wang, William R. Hersh JAMIA • 2020 TREC-COVID is an information retrieval (IR) shared task initiated to support clinicians and clinical research during the COVID-19 pandemic. IR for pandemics breaks many normal assumptions, which can be seen by examining nine important basic IR research…Ranking Significant Discrepancies in Clinical Reports
Sean MacAvaney, Arman Cohan, Nazli Goharian, Ross FiliceECIR • 2020 Medical errors are a major public health concern and a leading cause of death worldwide. Many healthcare centers and hospitals use reporting systems where medical practitioners write a preliminary medical report and the report is later reviewed, revised, and…Longformer: The Long-Document Transformer
Iz Beltagy, Matthew E. Peters, Arman CohanarXiv • 2020 Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce the Longformer with an attention mechanism that scales linearly…Just Add Functions: A Neural-Symbolic Language Model
David Demeter, Doug DowneyarXiv • 2019 Neural network language models (NNLMs) have achieved ever-improving accuracy due to more sophisticated architectures and increasing amounts of training data. However, the inductive bias of these models (formed by the distributional hypothesis of language…Pretrained Language Models for Sequential Sentence Classification
Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. WeldEMNLP • 2019 As a step toward better document-level understanding, we explore classification of a sequence of sentences into their corresponding categories, a task that requires understanding sentences in context of the document. Recent successful models for this task…SciBERT: A Pretrained Language Model for Scientific Text
Iz Beltagy, Kyle Lo, Arman CohanEMNLP • 2019 Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific…