Research - Papers
Explore a selection of our published work on a variety of key research challenges in AI.
Neural Extractive Search
Domain experts often need to extract structured information from large corpora. We advocate for a search paradigm called “extractive search”, in which a search query is enriched with capture-slots,…
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills
Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning. In this work, we propose to…
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths through Question Decomposition
Recent efforts to create challenge benchmarks that test the abilities of natural language understanding models have largely depended on human annotations. In this work, we introduce the “Break,…
Measuring and Improving Consistency in Pretrained Language Models
Consistency of a model — that is, the invariance of its behavior under meaning-preserving alternations in its input — is a highly desirable property in natural language processing. In this paper we…
Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand?
Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can ever “understand”…
Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes
We explore few-shot learning (FSL) for relation classification (RC). Focusing on the realistic scenario of FSL, in which a test instance might not belong to any of the target categories…
Memory-efficient Transformers via Top-k Attention
Following the success of dot-product attention in Transformers, numerous approximations have been recently proposed to address its quadratic complexity with respect to the input length. While these…
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing
The de-facto standard decoding method for semantic parsing in recent years has been to autoregressively decode the abstract syntax tree of the target program using a top-down depth-first traversal.…
MULTIMODALQA: COMPLEX QUESTION ANSWERING OVER TEXT, TABLES AND IMAGES
When answering complex questions, people can seamlessly combine information from visual, textual and tabular sources. While interest in models that reason over multiple pieces of evidence has surged…
BERTese: Learning to Speak to BERT
Large pre-trained language models have been shown to encode large amounts of world and commonsense knowledge in their parameters, leading to substantial interest in methods for extracting that…