Research - Papers
Explore a selection of our published work on a variety of key research challenges in AI.
Efficient Methods for Natural Language Processing: A Survey
Getting the most out of limited resources allows advances in natural language processing (NLP) research and practice while being con-servative with resources. Those resources may be data, time,…
MetaICL: Learning to Learn In Context
We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set…
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open-domain question answering and fact verification. These models are…
Robust fine-tuning of zero-shot models
Large pre-trained models such as CLIP or ALIGN offer consistent accuracy across a range of data distributions when performing zero-shot inference (i.e., without fine-tuning on a specific dataset).…
Noisy Channel Language Model Prompting for Few-Shot Text Classification
We introduce a noisy channel approach for language model prompting in few-shot text classification. Instead of computing the likelihood of the label given the input (referred as direct models),…
FaVIQ: FAct Verification from Information-seeking Questions
Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic real-world claims. Existing…
Impact of Warmer Sea Surface Temperature on the Global Pattern of Intense Convection: Insights From a Global Storm Resolving Model
Intense convection (updrafts exceeding 10 m s−1) plays an essential role in severe weather and Earth's energy balance. Despite its importance, how the global pattern of intense convection changes in…
Linear Adversarial Concept Erasure
We formulate the problem of identifying and erasing a linear subspace that corresponds to a given concept, in order to prevent linear predictors from recovering the concept. We model this problem as…
Dyna-bAbI: unlocking bAbI’s potential with dynamic synthetic benchmarking
While neural language models often perform surprisingly well on natural language understanding (NLU) tasks, their strengths and limitations remain poorly understood. Controlled synthetic tasks are…
Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback
Large language models (LMs), while power-ful, are not immune to mistakes, but can be difficult to retrain. Our goal is for an LM to continue to improve after deployment, without retraining, using…