Skip to main content ->
Ai2

Research - Papers

Explore a selection of our published work on a variety of key research challenges in AI.

Filter papers

Promoting Graph Awareness in Linearized Graph-to-Text Generation

Alexander M. HoyleAna MarasovićNoah A. Smith
2021
Findings of ACL

Generating text from structured inputs, such as meaning representations or RDF triples, has often involved the use of specialized graphencoding neural networks. However, recent applications of… 

Explaining NLP Models via Minimal Contrastive Editing (MiCE)

Alexis RossAna MarasovićMatthew E. Peters
2021
Findings of ACL

Humans give contrastive explanations that explain why an observed event happened rather than some other counterfactual event (the contrast case). Despite the important role that contrastivity plays… 

Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies

Mor GevaDaniel KhashabiElad SegalJonathan Berant
2021
TACL

A key limitation in current datasets for multi-hop reasoning is that the required steps for answering the question are mentioned in it explicitly. In this work, we introduce STRATEGYQA, a question… 

Few-Shot Question Answering by Pretraining Span Selection

Ori RamYuval KirstainJonathan BerantOmer Levy
2021
ACL

In a number of question answering (QA) benchmarks, pretrained models have reached human parity through fine-tuning on an order of 100,000 annotated questions and answers. We explore the more… 

Effective Attention Sheds Light On Interpretability

Kaiser Sun and Ana Marasović
2021
Findings of ACL

An attention matrix of a transformer selfattention sublayer can provably be decomposed into two components and only one of them (effective attention) contributes to the model output. This leads us… 

Neural Extractive Search

Shaul RavfogelHillel Taub-TabibYoav Goldberg
2021
ACL • Demo Track

Domain experts often need to extract structured information from large corpora. We advocate for a search paradigm called “extractive search”, in which a search query is enriched with capture-slots,… 

Shortformer: Better Language Modeling using Shorter Inputs

Ofir PressNoah A. SmithM. Lewis
2021
ACL

We explore the benefits of decreasing the input length of transformers. First, we show that initially training the model on short subsequences, before moving on to longer ones, both reduces overall… 

PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World

Rowan ZellersAri HoltzmanMatthew E. PetersYejin Choi
2021
ACL

We propose PIGLeT: a model that learns physical commonsense knowledge through interaction, and then uses this knowledge to ground language. We factorize PIGLeT into a physical dynamics model, and a… 

All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text

Elizabeth ClarkTal AugustSofia SerranoNoah A. Smith
2021
ACL

Human evaluations are typically considered the gold standard in natural language generation, but as models' fluency improves, how well can evaluators detect and judge machine-generated text? We run… 

How effective is BERT without word ordering? Implications for language understanding and data privacy

Jack HesselAlexandra Schofield
2021
ACL

Ordered word sequences contain the rich structures that define language. However, it’s often not clear if or how modern pretrained language models utilize these structures. We show that the token…