Research - Papers
Explore a selection of our published work on a variety of key research challenges in AI.
Efficient Methods for Natural Language Processing: A Survey
Getting the most out of limited resources allows advances in natural language processing (NLP) research and practice while being con-servative with resources. Those resources may be data, time,…
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open-domain question answering and fact verification. These models are…
FaVIQ: FAct Verification from Information-seeking Questions
Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic real-world claims. Existing…
MetaICL: Learning to Learn In Context
We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set…
Noisy Channel Language Model Prompting for Few-Shot Text Classification
We introduce a noisy channel approach for language model prompting in few-shot text classification. Instead of computing the likelihood of the label given the input (referred as direct models),…
Robust fine-tuning of zero-shot models
Large pre-trained models such as CLIP or ALIGN offer consistent accuracy across a range of data distributions when performing zero-shot inference (i.e., without fine-tuning on a specific dataset).…
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection
Warning : this paper discusses and contains content that is offensive or upsetting. The perceived toxicity of language can vary based on someone’s identity and beliefs, but this variation is often…
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
Natural language processing researchers have identified limitations of evaluation methodology for generation tasks, with new questions raised about the validity of automatic metrics and of…
DEMix Layers: Disentangling Domains for Modular Language Modeling
We introduce a new domain expert mixture (DEMIX) layer that enables conditioning a language model (LM) on the domain of the input text. A DEMIX layer is a collection of expert feedforward networks,…
Efficient Hierarchical Domain Adaptation for Pretrained Language Models
The remarkable success of large language models has been driven by dense models trained on massive unlabeled, unstructured corpora. These corpora typically contain text from diverse, heterogeneous…