How a Georgia Tech team used the open Olmo stack to trace social reasoning

Glenn Matlin, a computer-science PhD candidate at Georgia Tech, and his co-author Chandreyi (Zini) Chakraborty have spent years tracing the roots of a language model’s social reasoning—its ability to interpret people’s beliefs, emotions, intentions, and everyday moral choices. "We set out with one main question: where does [it] come from?” says Chakraborty.

Matlin and Chakraborty designed a study around influence functions, a technique that estimates how much any single training document – an article, post, or page a model trained on – shaped how the model answered a particular benchmark question. Add those estimates up across the data, and you can see which kinds of writing – literary, technical, and so on – support a given ability in the model. But that holds only if you can confirm the model actually trained on the documents in question.

It’s the kind of question they could only answer with a fully open model, which is why they chose to work with Olmo 3. Olmo 3 is one of the few LLMs released in full—its training corpora, checkpoints, and evaluation tools are all public alongside the more commonly released model weights. 

“The vision for my research is to create a map that connects model behavior back to the kinds of training data or dynamics that may have helped produce it,” Matlin says.

Matlin and Chakraborty’s study uses five open pieces of the Olmo ecosystem: Olmo 3, Dolma 3 (the data Olmo 3 was trained on), WebOrganizer (a labeling system for sorting datasets like Dolma into different categories by topic and format), OlmoEval (which supplies benchmarks to measure model abilities), and OLMES (a shared benchmark scoring standard). 

Dolma 3 contains about 1.26 billion documents—too many to analyze one by one. So the team sampled about 5.68 million, stratified across Dolma 3’s 576 document categories, and measured how much each one influenced Olmo 3's answers on a given benchmark, combining the results into a single score per category. They repeated this for four benchmarks: SocialIQA and ARC-Challenge to test reasoning, and MMLU's social-science and STEM sets to test factual knowledge. 

That gave each benchmark its own profile—the document categories that most shaped the model's answers on that evaluation. 

Comparing the four profiles, the team found a less intuitive split than simply “social” versus “scientific.” Social-science knowledge looked much more like STEM knowledge and reasoning than it did social reasoning. SocialIQA was the outlier: Olmo 3’s answers there leaned unusually heavily on narrative, interpersonal writing – Dolma categories such as literature, social life, customer support, and Q&A threads – while the other benchmarks drew more heavily on expository material such as technical documentation and science writing.

“Categories that were dialogue-rich and high in social and affective language influenced social reasoning more strongly than any other benchmark," says Chakraborty. 

But the pattern wasn't limited to social reasoning. Dialogue-rich, interpersonal writing was also more influential on both reasoning benchmarks than on the knowledge benchmarks, suggesting that this kind of data may contribute to reasoning in ways that go beyond simply teaching a model about social situations.

Matlin and Chakraborty then ran a causal test: if they deleted the data that most shaped an ability, would the model get worse at it? They had Olmo 3 "unlearn" the most influential documents in Dolma's literature category, which was the category most tied to Olmo 3’s social reasoning. Its SocialIQA score dropped further than when they removed random documents from that category. 

That points to a possible use for this kind of capability tracing beyond explaining model behavior after the fact. If researchers can identify which parts of a training corpus are actually supporting a particular ability, they can test how enriching, reweighting, or removing that data affects the model. The results don’t amount to a simple recipe – e.g., “add more literature to get better social reasoning” – but they could help researchers make training-data decisions based on what different kinds of data appear to contribute to.

Until now, tracing a capability back through a model's entire training dataset has mostly happened inside large labs that don’t make their data, model snapshots, and evaluation tools public. Ai2’s open artifacts enabled Matlin's team to run that kind of study as independent researchers.

"AI safety researchers need to do work that challenges standard assumptions about language models and do investigations about entirely novel methods to audit model behavior," Matlin says. "That would not be possible if researchers had to depend entirely on private labs for model artifacts. Ai2 did not simply release a model—it released enough of the scientific stack for an external team to investigate the model and reproduce it from scratch as needed.”

The researchers’ code and detailed experimental results are available from their project website, as well as their Discord server.