Viewing 1-10 of 30 papers
  • Abductive Commonsense Reasoning

    Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Scott Wen-tau Yih, Yejin ChoiICLR2020Abductive reasoning is inference to the most plausible explanation. For example, if Jenny finds her house in a mess when she returns from work, and remembers that she left a window open, she can hypothesize that a thief broke into her house and caused the mess, as the most plausible explanation… more
  • BERT for Coreference Resolution: Baselines and Analysis

    Mandar Joshi, Omer Levy, Daniel S. Weld, Luke ZettlemoyerEMNLP2019We apply BERT to coreference resolution, achieving strong improvements on the OntoNotes (+3.9 F1) and GAP (+11.5 F1) benchmarks. A qualitative analysis of model predictions indicates that, compared to ELMo and BERT-base, BERT-large is particularly better at distinguishing between related but… more
  • Pretrained Language Models for Sequential Sentence Classification

    Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. WeldEMNLP2019As a step toward better document-level understanding, we explore classification of a sequence of sentences into their corresponding categories, a task that requires understanding sentences in context of the document. Recent successful models for this task have used hierarchical models to… more
  • SciBERT: A Pretrained Language Model for Scientific Text

    Iz Beltagy, Kyle Lo, Arman CohanEMNLP2019Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. SciBERT leverages unsupervised… more
  • SpanBERT: Improving Pre-training by Representing and Predicting Spans

    Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, Omer LevyEMNLP2019We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text. Our approach extends BERT by (1) masking contiguous random spans, rather than random tokens, and (2) training the span boundary representations to predict the entire content of the masked span… more
  • GrapAL: Connecting the Dots in Scientific Literature

    Christine Betts, Joanna Power, Waleed AmmarACL2019We introduce GrapAL (Graph database of Academic Literature), a versatile tool for exploring and investigating a knowledge base of scientific literature, that was semi-automatically constructed using NLP methods. GrapAL satisfies a variety of use cases and information needs requested by researchers… more
  • ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

    Mark Neumann, Daniel King, Iz Beltagy, Waleed AmmarACL • BioNLP Workshop2019Despite recent advances in natural language processing, many statistical models for processing text perform extremely poorly under domain shift. Processing biomedical and clinical text is a critically important application area of natural language processing, for which there are few robust… more
  • CEDR: Contextualized Embeddings for Document Ranking

    Sean MacAvaney, Andrew Yates, Arman Cohan, Nazli GoharianSIGIR2019Although considerable attention has been given to neural ranking architectures recently, far less attention has been paid to the term representations that are used as input to these models. In this work, we investigate how two pretrained contextualized language models (ELMo and BERT) can be… more
  • Ontology-Aware Clinical Abstractive Summarization

    Sean MacAvaney, Sajad Sotudeh, Arman Cohan, Nazli Goharian, Ish Talati, Ross W. FiliceSIGIR2019Automatically generating accurate summaries from clinical reports could save a clinician's time, improve summary coverage, and reduce errors. We propose a sequence-to-sequence abstractive summarization model augmented with domain-specific ontological information to enhance content selection and… more
  • Quantifying Sex Bias in Clinical Studies at Scale With Automated Data Extraction

    Sergey Feldman, Waleed Ammar, Kyle Lo, Elly Trepman, Madeleine van Zuylen, Oren EtzioniJAMA2019Importance: Analyses of female representation in clinical studies have been limited in scope and scale. Objective: To perform a large-scale analysis of global enrollment sex bias in clinical studies. Design, Setting, and Participants: In this cross-sectional study, clinical studies from published… more