Viewing 1-10 of 103 papers
  • Adversarial Filters of Dataset Biases

    Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew E. Peters, Ashish Sabharwal, Yejin Choi arXiv2020Large neural models have demonstrated humanlevel performance on language and vision benchmarks such as ImageNet and Stanford Natural Language Inference (SNLI). Yet, their performance degrades considerably when tested on adversarial or out-of-distribution samples. This raises the question of whether… more
  • Transformers as Soft Reasoners over Language

    Peter Clark, Oyvind Tafjord, Kyle RichardsonarXiv2020AI has long pursued the goal of having systems reason over explicitly provided knowledge, but building suitable representations has proved challenging. Here we explore whether transformers can similarly learn to reason (or emulate reasoning), but using rules expressed in language, thus bypassing a… more
  • QASC: A Dataset for Question Answering via Sentence Composition

    Tushar Khot, Peter Clark, Michal Guerquin, Paul Edward Jansen, Ashish Sabharwal AAAI2020Composing knowledge from multiple pieces of texts is a key challenge in multi-hop question answering. We present a multi-hop reasoning dataset, Question Answering via Sentence Composition (QASC), that requires retrieving facts from a large corpus and composing them to answer a multiple-choice… more
  • Probing Natural Language Inference Models through Semantic Fragments

    Kyle Richardson, Hai Na Hu, Lawrence S. Moss, Ashish SabharwalAAAI2020Do state-of-the-art models for language understanding already have, or can they easily learn, abilities such as boolean coordination, quantification, conditionals, comparatives, and monotonicity reasoning (i.e., reasoning about word substitutions in sentential contexts)? While such phenomena are… more
  • MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity

    Hai Hu, Qi Chen, Kyle Richardson, Atreyee Mukherjee, Lawrence S. Moss, Sandra Kübler SCIL2020We present a new logic-based inference engine for natural language inference (NLI) called MonaLog, which is based on natural logic and the monotonicity calculus. In contrast to existing logic-based approaches, our system is intentionally designed to be as lightweight as possible, and operates using… more
  • What Does My QA Model Know? Devising Controlled Probes using Expert Knowledge

    Kyle Richardson, Ashish SabharwalarXiv2019Open-domain question answering (QA) is known to involve several underlying knowledge and reasoning challenges, but are models actually learning such knowledge when trained on benchmark tasks? To investigate this, we introduce several new challenge tasks that probe whether state-of-theart QA models… more
  • Approximating the Permanent by Sampling from Adaptive Partitions

    Jonathan Kuck, Tri Dao, Hamid Rezatofighi, Ashish Sabharwal, Stefano ErmonarXiv2019Computing the permanent of a non-negative matrix is a core problem with practical applications ranging from target tracking to statistical thermodynamics. However, this problem is also #P-complete, which leaves little hope for finding an exact solution that can be computed efficiently. While the… more
  • Not All Claims are Created Equal: Choosing the Right Approach to Assess Your Hypotheses

    Erfan Sadeqi Azer, Daniel Khashabi, Ashish Sabharwal, Dan RotharXiv2019Empirical research in Natural Language Processing (NLP) has adopted a narrow set of principles for assessing hypotheses, relying mainly on p-value computation, which suffers from several known issues. While alternative proposals have been well-debated and adopted in other fields, they remain rarely… more
  • What's Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering

    Tushar Khot, Ashish Sabharwal, Peter ClarkEMNLP2019Multi-hop textual question answering requires combining information from multiple sentences. We focus on a natural setting where, unlike typical reading comprehension, only partial information is provided with each question. The model must retrieve and use additional knowledge to correctly answer… more
  • Reasoning Over Paragraph Effects in Situations

    Kevin Lin, Oyvind Tafjord, Peter Clark, Matt GardnerEMNLP • MRQA Workshop2019A key component of successfully reading a passage of text is the ability to apply knowledge gained from the passage to a new situation. In order to facilitate progress on this kind of reading, we present ROPES, a challenging benchmark for reading comprehension targeting Reasoning Over Paragraph… more