Skip to main content ->
Ai2

Research - Papers

Explore a selection of our published work on a variety of key research challenges in AI.

Filter papers

MetaICL: Learning to Learn In Context

Sewon MinM. LewisLuke ZettlemoyerHannaneh Hajishirzi
2022
NAACL

We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set… 

Noisy Channel Language Model Prompting for Few-Shot Text Classification

Sewon MinMichael LewisHannaneh HajishirziLuke Zettlemoyer
2022
ACL

We introduce a noisy channel approach for language model prompting in few-shot text classification. Instead of computing the likelihood of the label given the input (referred as direct models),… 

Robust fine-tuning of zero-shot models

Mitchell WortsmanGabriel IlharcoMike LiLudwig Schmidt
2022
CVPR

Large pre-trained models such as CLIP or ALIGN offer consistent accuracy across a range of data distributions when performing zero-shot inference (i.e., without fine-tuning on a specific dataset).… 

Impact of Warmer Sea Surface Temperature on the Global Pattern of Intense Convection: Insights From a Global Storm Resolving Model

K. ChengL. HarrisC. BrethertonS. Fueglistaler
2022
Geophysical Research Letters

Intense convection (updrafts exceeding 10 m s−1) plays an essential role in severe weather and Earth's energy balance. Despite its importance, how the global pattern of intense convection changes in… 

Linear Adversarial Concept Erasure

Shauli RavfogelMichael TwitonYoav GoldbergRyan Cotterell
2022
ICML

We formulate the problem of identifying and erasing a linear subspace that corresponds to a given concept, in order to prevent linear predictors from recovering the concept. We model this problem as… 

Dyna-bAbI: unlocking bAbI’s potential with dynamic synthetic benchmarking

Ronen TamariKyle RichardsonAviad Sar-ShalomDafna Shahaf
2022
SEM

While neural language models often perform surprisingly well on natural language understanding (NLU) tasks, their strengths and limitations remain poorly understood. Controlled synthetic tasks are… 

Aligning to Social Norms and Values in Interactive Narratives

Prithviraj AmmanabroluLiwei JiangMaarten SapYejin Choi
2022
NAACL

We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games—environments wherein an agent perceives and interacts with a… 

Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback

Niket TandonAman MadaanPeter ClarkYiming Yang
2022
Findings of NAACL

Large language models (LMs), while power-ful, are not immune to mistakes, but can be difficult to retrain. Our goal is for an LM to continue to improve after deployment, without retraining, using… 

A Dataset for N-ary Relation Extraction of Drug Combinations

Aryeh TiktinskyVijay ViswanathanDanna NiezniYoav Goldberg
2022
NAACL

Combination therapies have become the standard of care for diseases such as cancer, tuberculosis, malaria and HIV. However, the combinatorial set of available multi-drug treatments creates a… 

Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection

Maarten SapSwabha SwayamdiptaLaura ViannaNoah A. Smith
2022
NAACL

Warning : this paper discusses and contains content that is offensive or upsetting. The perceived toxicity of language can vary based on someone’s identity and beliefs, but this variation is often…