Papers

Learn more about AI2's Lasting Impact Award
Viewing 21-30 of 99 papers
  • Few-Shot Question Answering by Pretraining Span Selection

    Ori Ram, Yuval Kirstain, Jonathan Berant, A. Globerson, Omer LevyACL2021 In a number of question answering (QA) benchmarks, pretrained models have reached human parity through fine-tuning on an order of 100,000 annotated questions and answers. We explore the more realistic few-shot setting, where only a few hundred training…
  • Neural Extractive Search

    Shaul Ravfogel, Hillel Taub-Tabib, Yoav GoldbergACL • Demo Track2021 Domain experts often need to extract structured information from large corpora. We advocate for a search paradigm called “extractive search”, in which a search query is enriched with capture-slots, to allow for such rapid extraction. Such an extractive search…
  • Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

    Ori Yoran, Alon Talmor, Jonathan BerantarXiv2021 Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning. In this work, we propose to leverage semi-structured tables, and automatically generate at…
  • Break, Perturb, Build: Automatic Perturbation of Reasoning Paths through Question Decomposition

    Mor Geva, Tomer Wolfson, Jonathan BerantTACL 2021 Recent efforts to create challenge benchmarks that test the abilities of natural language understanding models have largely depended on human annotations. In this work, we introduce the “Break, Perturb, Build” (BPB) framework for automatic reasoning-oriented…
  • Measuring and Improving Consistency in Pretrained Language Models

    Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav GoldbergTACL2021 Consistency of a model — that is, the invariance of its behavior under meaning-preserving alternations in its input — is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs…
  • Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand?

    William Merrill, Yoav Goldberg, Roy Schwartz, Noah A. SmithTACL2021 Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can ever “understand” raw text without access to some form of grounding. We…
  • Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes

    Ofer Sabo, Yanai Elazar, Yoav Goldberg, Ido DaganTACL2021 We explore few-shot learning (FSL) for relation classification (RC). Focusing on the realistic scenario of FSL, in which a test instance might not belong to any of the target categories (none-of-the-above, [NOTA]), we first revisit the recent popular dataset…
  • Memory-efficient Transformers via Top-k Attention

    Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonathan BerantarXiv2021 Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length. While these variants are memory and compute efficient, it is not possible…
  • SmBoP: Semi-autoregressive Bottom-up Semantic Parsing

    Ohad Rubin and Jonathan BerantNAACL2021 The de-facto standard decoding method for semantic parsing in recent years has been to autoregressively decode the abstract syntax tree of the target program using a top-down depth-first traversal. In this work, we propose an alternative approach: a Semi…
  • MULTIMODALQA: COMPLEX QUESTION ANSWERING OVER TEXT, TABLES AND IMAGES

    Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, Jonathan BerantICLR2021 When answering complex questions, people can seamlessly combine information from visual, textual and tabular sources. While interest in models that reason over multiple pieces of evidence has surged in recent years, there has been relatively little work on…