Skip to main content ->
Ai2

Research - Papers

Explore a selection of our published work on a variety of key research challenges in AI.

Filter papers

Longformer: The Long-Document Transformer

Iz BeltagyMatthew E. PetersArman Cohan
2020
arXiv

Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce the… 

Evaluating NLP Models via Contrast Sets

M.GardnerY.ArtziV.Basmovaet.al
2020
arXiv

Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading:… 

Multi-View Learning for Vision-and-Language Navigation

Qiaolin XiaXiujun LiChunyuan LiNoah A. Smith
2020
arXiv

Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified.… 

Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping

Jesse DodgeGabriel IlharcoRoy SchwartzNoah A. Smith
2020
arXiv

Fine-tuning pretrained contextual word embedding models to supervised downstream tasks has become commonplace in natural language processing. This process, however, is often brittle: even with the… 

Analyzing Compositionality in Visual Question Answering

Sanjay SubramanianSameer SinghMatt Gardner
2019
NeurIPS • ViGIL Workshop

Since the release of the original Visual Question Answering (VQA) dataset, several newer datasets for visual reasoning have been introduced, often with the express intent of requiring systems to… 

Evaluating Question Answering Evaluation

Anthony ChenGabriel StanovskySameer SinghMatt Gardner
2019
EMNLP • MRQA Workshop

As the complexity of question answering (QA) datasets evolve, moving away from restricted formats like span extraction and multiple-choice (MC) to free-form answer generation, it is imperative to… 

On Making Reading Comprehension More Comprehensive

Matt GardnerJonathan BerantHannaneh HajishirziSewon Min
2019
EMNLP • MRQA Workshop

Machine reading comprehension, the task of evaluating a machine’s ability to comprehend a passage of text, has seen a surge in popularity in recent years. There are many datasets that are targeted… 

ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension

Dheeru DuaAnanth GottumukkalaAlon TalmorMatt Gardner
2019
EMNLP • MRQA Workshop

Reading comprehension is one of the crucial tasks for furthering research in natural language understanding. A lot of diverse reading comprehension datasets have recently been introduced to study… 

AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models

Eric WallaceJens TuylsJunlin WangSameer Singh
2019
EMNLP

Neural NLP models are increasingly accurate but are imperfect and opaque---they break in counterintuitive ways and leave end users puzzled at their behavior. Model interpretation methods ameliorate… 

Do NLP Models Know Numbers? Probing Numeracy in Embeddings

Eric WallaceYizhong WangSujian LiMatt Gardner
2019
EMNLP

The ability to understand and work with numbers (numeracy) is critical for many complex reasoning tasks. Currently, most NLP models treat numbers in text in the same way as other tokens---they embed…