Papers
See AI2's Award Winning Papers
Learn more about AI2's Lasting Impact Award
Viewing 671-680 of 991 papers
Longformer: The Long-Document Transformer
Iz Beltagy, Matthew E. Peters, Arman CohanarXiv • 2020 Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce the Longformer with an attention mechanism that scales linearly…TuringAdvice: A Generative and Dynamic Evaluation of Language Use
Rowan Zellers, Ari Holtzman, Elizabeth Clark, Lianhui Qin, Ali Farhadi, Yejin ChoiNAACL • 2020 We propose TuringAdvice, a new challenge task and dataset for language understanding models. Given a written situation that a real person is currently facing, a model must generate helpful advice in natural language. Our evaluation framework tests a…Evaluating NLP Models via Contrast Sets
M.Gardner, Y.Artzi, V.Basmova, J.Berant, B.Bogin, S.Chen, P.Dasigi, D.Dua, Y.Elazar, A.Gottumukkala, N.Gupta, H.Hajishirzi, G.Ilharco, D.Khashabi, K.Lin, J.Liu, N.Liu, P.Mulcaire, Q.Ning, S.Singh, N.Smith, S.Subramanian, R.Tsarfaty, E.Wallace, et.alarXiv • 2020 Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on…Soft Threshold Weight Reparameterization for Learnable Sparsity
Aditya Kusupati, Vivek Ramanujan, Raghav Somani, Mitchell Wortsman, Prateek Jain, Sham Kakade, Ali FarhadiICML • 2020 Sparsity in Deep Neural Networks (DNNs) is studied extensively with the focus of maximizing prediction accuracy given an overall parameter budget. Existing methods rely on uniform or heuristic non-uniform sparsity budgets which have sub-optimal layer-wise…Differentiable Scene Graphs
Moshiko Raboh, Roei Herzig, Gal Chechik, Jonathan Berant, Amir Globerson WACV • 2020 Understanding the semantics of complex visual scenes involves perception of entities and reasoning about their relations. Scene graphs provide a natural representation for these tasks, by assigning labels to both entities (nodes) and relations (edges…Multi-View Learning for Vision-and-Language Navigation
Qiaolin Xia, Xiujun Li, Chunyuan Li, Yonatan Bisk, Zhifang Sui, Yejin Choi, Noah A. SmitharXiv • 2020 Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified. In this paper, we present a novel training paradigm, Learn…Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, Noah A. Smith arXiv • 2020 Fine-tuning pretrained contextual word embedding models to supervised downstream tasks has become commonplace in natural language processing. This process, however, is often brittle: even with the same hyperparameter values, distinct random seeds can lead to…WinoGrande: An Adversarial Winograd Schema Challenge at Scale
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI • 2020 The Winograd Schema Challenge (WSC), proposed by Levesque et al. (2011) as an alternative to the Turing Test, was originally designed as a pronoun resolution problem that cannot be solved based on statistical patterns in large text corpora. However, recent…PIQA: Reasoning about Physical Commonsense in Natural Language
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, Yejin ChoiAAAI • 2020 To apply eyeshadow without a brush, should I use a cotton swab or a toothpick? Questions requiring this kind of physical commonsense pose a challenge to today's natural language understanding systems. While recent pretrained models (such as BERT) have made…Probing Natural Language Inference Models through Semantic Fragments
Kyle Richardson, Hai Na Hu, Lawrence S. Moss, Ashish SabharwalAAAI • 2020 Do state-of-the-art models for language understanding already have, or can they easily learn, abilities such as boolean coordination, quantification, conditionals, comparatives, and monotonicity reasoning (i.e., reasoning about word substitutions in…