Papers

Learn more about AI2's Lasting Impact Award
Viewing 741-750 of 1022 papers
  • PaLM: A Hybrid Parser and Language Model

    Hao Peng, Roy Schwartz, Noah A. SmithEMNLP2019 We present PaLM, a hybrid parser and neural language model. Building on an RNN language model, PaLM adds an attention layer over text spans in the left context. An unsupervised constituency parser can be derived from its attention weights, using a greedy…
  • Pretrained Language Models for Sequential Sentence Classification

    Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. WeldEMNLP2019 As a step toward better document-level understanding, we explore classification of a sequence of sentences into their corresponding categories, a task that requires understanding sentences in context of the document. Recent successful models for this task…
  • QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions

    Oyvind Tafjord, Matt Gardner, Kevin Lin, Peter ClarkEMNLP2019 We introduce the first open-domain dataset, called QuaRTz, for reasoning about textual qualitative relationships. QuaRTz contains general qualitative statements, e.g., "A sunscreen with a higher SPF protects the skin longer.", twinned with 3864 crowdsourced…
  • Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning

    Pradeep Dasigi, Nelson F. Liu, Ana Marasovic, Noah A. Smith, Matt GardnerEMNLP2019 Machine comprehension of texts longer than a single sentence often requires coreference resolution. However, most current reading comprehension benchmarks do not contain complex coreferential phenomena and hence fail to evaluate the ability of models to…
  • RNN Architecture Learning with Sparse Regularization

    Jesse Dodge, Roy Schwartz, Hao Peng, Noah A. SmithEMNLP2019 Neural models for NLP typically use large numbers of parameters to reach state-of-the-art performance, which can lead to excessive memory usage and increased runtime. We present a structure learning method for learning sparse, parameter-efficient NLP models…
  • SciBERT: A Pretrained Language Model for Scientific Text

    Iz Beltagy, Kyle Lo, Arman CohanEMNLP2019 Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific…
  • Show Your Work: Improved Reporting of Experimental Results

    Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, Noah A. SmithEMNLP2019 Research in natural language processing proceeds, in part, by demonstrating that new models achieve superior performance (e.g., accuracy) on held-out test data, compared to previous results. In this paper, we demonstrate that test-set performance scores alone…
  • Social IQA: Commonsense Reasoning about Social Interactions

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, Yejin ChoiEMNLP2019 We introduce Social IQa, the first largescale benchmark for commonsense reasoning about social situations. Social IQa contains 38,000 multiple choice questions for probing emotional and social intelligence in a variety of everyday situations (e.g., Q: "Jordan…
  • SpanBERT: Improving Pre-training by Representing and Predicting Spans

    Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, Omer LevyEMNLP2019 We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text. Our approach extends BERT by (1) masking contiguous random spans, rather than random tokens, and (2) training the span boundary representations to…
  • Topics to Avoid: Demoting Latent Confounds in Text Classification

    Sachin Kumar, Shuly Wintner, Noah A. Smith, Yulia TsvetkovEMNLP2019 Despite impressive performance on many text classification tasks, deep neural networks tend to learn frequent superficial patterns that are specific to the training data and do not always generalize well. In this work, we observe this limitation with respect…