Ai2 Newsletter
October 2024
Top story - Meet Molmo, our open, state-of-the-art multimodal models
"Ai2’s Molmo shows open source can meet, and beat, closed multimodal models." — Devin Coldewey for TechCrunch.
We've just introduced our latest family of models to the world: meet Molmo!
Molmo doesn't just understand multimodal data — it acts on it, enabling rich interactions in both the physical and virtual worlds. Molmo can point at things in an image, which empowers web agents to navigate websites.
Open doesn't mean sacrificing performance — Molmo outperforms GPT-4o, Gemini 1.5 Pro, and Claude 3.5 across 11 benchmarks at a much more efficient size.
Investigating pretraining dynamics and stability with OLMo checkpoints
With OLMo, we hope to contribute to the open science of LM pretraining to provide a foundation for open-source pretraining efforts. This blog post covers a simple but illustrative investigation into the evolution of parameter and activation magnitudes throughout pretraining checkpoints from OLMo 7B 0724.
New benchmark on setting up and executing tasks from research repositories
We introduce SUPER to evaluate LLMs' capabilities in autonomously running experiments from research repositories. Our benchmark comprises 45 end-to-end problems with annotated expert solutions,152 sub-problems derived from the expert set that focus on specific challenges, and 602 automatically generated problems for larger-scale development.
Join us!
Are you excited about accelerating science with AI? Semantic Scholar Research is hiring, and is especially interested in the following areas:
- Human-AI collaboration
- Agentic planning, reasoning, and learning
- Model distillation and specialization
- Retrieval and RAG systems (for the Research Intern position)
If that's you, check out our 2025 openings and apply by Nov. 1st for full consideration!
- Research Scientist
- Young Investigator (postdoc)
- Research Intern