Skip to main content ->
Ai2

Research - Papers

Explore a selection of our published work on a variety of key research challenges in AI.

Filter papers

Few-Shot Question Answering by Pretraining Span Selection

Ori RamYuval KirstainJonathan BerantOmer Levy
2021
ACL

In a number of question answering (QA) benchmarks, pretrained models have reached human parity through fine-tuning on an order of 100,000 annotated questions and answers. We explore the more… 

How effective is BERT without word ordering? Implications for language understanding and data privacy

Jack HesselAlexandra Schofield
2021
ACL

Ordered word sequences contain the rich structures that define language. However, it’s often not clear if or how modern pretrained language models utilize these structures. We show that the token… 

Neural Extractive Search

Shaul RavfogelHillel Taub-TabibYoav Goldberg
2021
ACL • Demo Track

Domain experts often need to extract structured information from large corpora. We advocate for a search paradigm called “extractive search”, in which a search query is enriched with capture-slots,… 

PAWLS: PDF Annotation With Labels and Structure

Mark NeumannZejiang ShenSam Skjonsberg
2021
Demo • ACL

Adobe’s Portable Document Format (PDF) is a popular way of distributing view-only documents with a rich visual markup. This presents a challenge to NLP practitioners who wish to use the information… 

PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World

Rowan ZellersAri HoltzmanMatthew E. PetersYejin Choi
2021
ACL

We propose PIGLeT: a model that learns physical commonsense knowledge through interaction, and then uses this knowledge to ground language. We factorize PIGLeT into a physical dynamics model, and a… 

Promoting Graph Awareness in Linearized Graph-to-Text Generation

Alexander M. HoyleAna MarasovićNoah A. Smith
2021
Findings of ACL

Generating text from structured inputs, such as meaning representations or RDF triples, has often involved the use of specialized graphencoding neural networks. However, recent applications of… 

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

Julia KreutzerIsaac CaswellLisa WangMofetoluwa Adeyemi
2021
TACL

With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text datasets covering… 

Shortformer: Better Language Modeling using Shorter Inputs

Ofir PressNoah A. SmithM. Lewis
2021
ACL

We explore the benefits of decreasing the input length of transformers. First, we show that initially training the model on short subsequences, before moving on to longer ones, both reduces overall… 

Efficient Passage Retrieval with Hashing for Open-domain Question Answering

Ikuya YamadaAkari AsaiHanna Hajishirzi
2021
ACL

Most state-of-the-art open-domain question answering systems use a neural retrieval model to encode passages into continuous vectors and extract them from a knowledge source. However, such retrieval… 

Prompting Contrastive Explanations for Commonsense Reasoning Tasks

Bhargavi ParanjapeJulian MichaelMarjan GhazvininejadHanna Hajishirzi
2021
Findings of ACL

Many commonsense reasoning NLP tasks involve choosing between one or more possible answers to a question or prompt based on knowledge that is often implicit. Large pretrained language models (PLMs)…