Papers

Learn more about AI2's Lasting Impact Award
Viewing 11-20 of 106 papers
  • Inferring Implicit Relations with Language Models

    Uri Katz, Mor Geva, Jonathan BerantNAACL • UnImplicit2022 A prominent challenge for modern language understanding systems is the ability to answer implicit reasoning questions, where the required reasoning steps for answering the question are not mentioned in the text explicitly. In this work, we investigate why…
  • LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models

    Mor Geva, Avi Caciularu, Guy Dar, Paul Roit, Shoval Sadde, Micah Shlain, Bar Tamir, Yoav GoldbergarXiv2022 The opaque nature and unexplained behavior of transformer-based language models (LMs) have spurred a wide interest in interpreting their predictions. However, current interpretation methods mostly focus on probing models from outside, executing behavioral…
  • Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space

    Mor Geva, Avi Caciularu, Kevin Ro Wang, Yoav GoldbergarXiv2022 Transformer-based language models (LMs) are at the core of modern NLP, but their inter-nal prediction construction process is opaque and largely not understood. In this work, we make a substantial step towards unveiling this underlying prediction process, by…
  • Text-based NP Enrichment

    Yanai Elazar, Victoria Basmov, Yoav Goldberg, Reut TsarfatyTACL2022 Understanding the relations between entities denoted by NPs in text is a critical part of human-like natural language understanding. However, only a fraction of such relations is covered by NLP tasks and models nowadays. In this work, we establish the task of…
  • SCROLLS: Standardized CompaRison Over Long Language Sequences

    Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, Omer LevyarXiv2022 NLP benchmarks have largely focused on short texts, such as sentences and paragraphs, even though long texts comprise a considerable amount of natural language in the wild. We introduce SCROLLS, a suite of tasks that require reasoning over long texts. We…
  • CommonsenseQA 2.0: Exposing the Limits of AI through Gamification

    Alon Talmor, Ori Yoran, Ronan Le Bras, Chandrasekhar Bhagavatula, Yoav Goldberg, Yejin Choi, Jonathan Berant NeurIPS2021 Constructing benchmarks that test the abilities of modern natural language un1 derstanding models is difficult – pre-trained language models exploit artifacts in 2 benchmarks to achieve human parity, but still fail on adversarial examples and make 3 errors…
  • Achieving Model Robustness through Discrete Adversarial Training

    Maor Ivgi, Jonathan BerantEMNLP2021 Discrete adversarial attacks are symbolic perturbations to a language input that preserve the output label but lead to a prediction error. While such attacks have been extensively explored for the purpose of evaluating model robustness, their utility for…
  • Back to Square One: Bias Detection, Training and Commonsense Disentanglement in the Winograd Schema

    Yanai Elazar, Hongming Zhang, Yoav Goldberg, Dan RothEMNLP2021 The Winograd Schema (WS) has been proposed as a test for measuring commonsense capabilities of models. Recently, pre-trained language model-based approaches have boosted performance on some WS benchmarks but the source of improvement is still not clear. We…
  • Contrastive Explanations for Model Interpretability

    Alon Jacovi, Swabha Swayamdipta, Shauli Ravfogel, Yanai Elazar, Yejin Choi, Yoav GoldbergEMNLP2021 Contrastive explanations clarify why an event occurred in contrast to another. They are more inherently intuitive to humans to both produce and comprehend. We propose a methodology to produce contrastive explanations for classification models by modifying the…
  • Parameter Norm Growth During Training of Transformers

    William Merrill, Vivek Ramanujan, Yoav Goldberg, Roy Schwartz, Noah A. Smith EMNLP2021 The capacity of neural networks like the widely adopted transformer is known to be very high. Evidence is emerging that they learn successfully due to inductive bias in the training routine, typically some variant of gradient descent (GD). To better…