Skip to main content ->
Ai2

Research - Papers

Explore a selection of our published work on a variety of key research challenges in AI.

Filter papers

Plausibly Problematic Questions in Multiple-Choice Benchmarks for Commonsense Reasoning

Shramay PaltaNishant BalepurPeter RankelRachel Rudinger
2024
EMNLP Findings

Questions involving commonsense reasoning about everyday situations often admit many possible or plausible answers. In contrast, multiple-choice question (MCQ) benchmarks for commonsense reasoning… 

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging

Clara NaIan MagnussonAnanya Harsh JhaPradeep Dasigi
2024
EMNLP

Training data compositions for Large Language Models (LLMs) can significantly affect their downstream performance. However, a thorough data ablation study exploring large sets of candidate data… 

SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories

Ben BoginKejuan YangShashank GuptaTushar Khot
2024
EMNLP

Given that Large Language Models (LLMs) have made significant progress in writing code, can they now be used to autonomously reproduce results from research repositories? Such a capability would be… 

ComPO: Community Preferences for Language Model Personalization

Sachin KumarChan Young ParkYulia TsvetkovHanna Hajishirzi
2024
arXiv.org

Conventional algorithms for training language models (LMs) with human feedback rely on preferences that are assumed to account for an"average"user, disregarding subjectivity and finer-grained… 

CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization

Bodhisattwa Prasad MajumderBhavana Dalvi MishraPeter JansenPeter Clark
2024
COLM

Language agents have shown some ability to interact with an external environment, e.g., a virtual world such as ScienceWorld, to perform complex tasks, e.g., growing a plant, without the startup… 

IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback

Kevin PuK. FengTovi GrossmanPao Siangliulue
2024
arXiv.org

Research ideation involves broad exploring and deep refining ideas. Both require deep engagement with literature. Existing tools focus primarily on idea broad generation, yet offer little support… 

m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks

Zixian MaWeikai HuangJieyu ZhangRanjay Krishna
2024
ECCV

Real-world multi-modal problems are rarely solved by a single machine learning model, and often require multi-step computational plans that involve stitching several models. Tool-augmented LLMs hold… 

FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuning

Jiaheng HuRose HendrixAli FarhadiKiana Ehsan
2024
ICRA

In recent years, the Robotics field has initiated several efforts toward building generalist robot policies through large-scale multi-task Behavior Cloning. However, direct deployments of these… 

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models

Matt DeitkeChristopher ClarkSangho LeeAniruddha Kembhavi
2024
arXiv

Today's most advanced multimodal models remain proprietary. The strongest open-weight models rely heavily on synthetic data from proprietary VLMs to achieve good performance, effectively distilling… 

Application of the AI2 Climate Emulator to E3SMv2's global atmosphere model, with a focus on precipitation fidelity

James P. C. DuncanElynn WuJean-Christoph Golazand Christopher S. Bretherton
2024
Journal of Geophysical Research - Machine Learning

Can the current successes of global machine learning-based weather simulators be generalized beyond 2-week forecasts to stable and accurate multiyear runs? The recently developed AI2 Climate…