ARISTO

Build machines that read, learn and reason.

The Aristo Project aims to build systems that demonstrate a deep understanding of the world, integrating technologies for reading, learning, reasoning, and explanation.

A multiple choice question and reasoning explaining each answer

Our research integrates multiple AI technologies, including:

Natural language processing
Information extraction
Knowledge representation
Machine reasoning
Commonsense knowledge

Research Areas

Probing Reasoning with Language Models

Language models (LMs) have dominated much of AI recently. But what kind(s) of reasoning are they capable of? And how can they be taught to do more? We are developing analytical datasets to probe LMs and help answer these questions.

Learn More:

Probing Natural Language Inference Models through Semantic Fragments

Multihop Reasoning

Many questions require multiple pieces of information to be combined to arrive at an answer. We are developing new multihop models capable of identifying and combining relevant facts to answer such questions.

Learn More:

Repurposing Entailment for Multi-Hop Question Answering Tasks

QASC: A Dataset for Question Answering via Sentence Composition

Explanation

An intelligent system should not only answer questions correctly, but also be able to explain why its answers are correct. Such a capability is essential for practical acceptance of AI technology. It is also essential for the broader goals of communicating knowledge to a user, and receiving correction from the user when the system's answer is wrong.

Learn More:

What's Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering

Exploiting Explicit Paths for Multi-hop Reading Comprehension

Reasoning about Actions

A key aspect of intelligence is being able to reason about the dynamics of the world. This requires modeling what state the world might be in, and how different actions might affect that state. Such capabilities are essential for understanding what happens during a procedure or process, for planning, and for reasoning about "what if..." scenarios.

Learn More:

Reasoning about Actions and State Changes by Injecting Commonsense Knowledge

Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text

Demos

View All Aristo Demos

ModularQA

Modular QA answers questions by breaking them down into a series of smaller, more specific ones. This produces answers in a human-like way that's more explainable than black-box systems. | Aristo

ModularQA is a neuro-symbolic question-answering system that answers complex questions by asking a series of sub-questions to existing simpler QA systems or symbolic modules. It explains each of its reasoning steps in language, in terms of a simple question and its answer as produced by a simpler model or a math… more

Try the demo

ModularQA

Modular QA answers questions by breaking them down into a series of smaller, more specific ones. This produces answers in a human-like way that's more explainable than black-box systems. | Aristo

Try the demo

UnQover

Uncovering stereotypical biases via underspecified questions | Aristo

This work focuses specifically on identifying biases in question answering (QA) models. If these models are blindly deployed in real-life settings, the biases within these models could cause real harm, which raises the question; how extensive are social stereotypes in question-answering models?

Try the demo

UnQover

Uncovering stereotypical biases via underspecified questions | Aristo

Try the demo

Recent Papers

View All Aristo Papers

Explaining Answers with Entailment Trees
Bhavana Dalvi, Peter A. Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, Peter ClarkEMNLP • 2021 Our goal, in the context of open-domain textual question-answering (QA), is to explain answers by not just listing supporting textual evidence (“rationales”), but also showing how such evidence leads to the answer in a systematic way. If this could be done… more
GooAQ: Open Question Answering with Diverse Answer Types
Daniel Khashabi, Amos Ng, Tushar Khot, Ashish Sabharwal, Hanna Hajishirzi, Chris Callison-BurchFindings of EMNLP • 2021 While day-to-day questions come with a variety of answer types, the current questionanswering (QA) literature has failed to adequately address the answer diversity of questions. To this end, we present GOOAQ, a large-scale dataset with a variety of answer… more
proScript: Partially Ordered Scripts Generation
Keisuke Sakaguchi, Chandra Bhagavatula, R. L. Bras, Niket Tandon, P. Clark, Yejin ChoiFindings of EMNLP • 2021 Scripts standardized event sequences describing typical everyday activities have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information. However, to date they have proved hard to author or… more
Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?
Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal and Kai-Wei Chang ACL-IJCNLP • 2021 Is it possible to use natural language to intervene in a model’s behavior and alter its prediction in a desired way? We investigate the effectiveness of natural language interventions for reading-comprehension systems, studying this in the context of social… more
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference
Hai Hu, He Zhou, Zuoyu Tian, Yiwen Zhang, Yina Ma, Yanting Li, Yixin Nie, Kyle RichardsonFindings of ACL • 2021 Multilingual transformers (XLM, mT5) have been shown to have remarkable transfer skills in zero-shot settings. Most transfer studies, however, rely on automatically translated resources (XNLI, XQuAD), making it hard to discern the particular linguistic… more

View All Aristo Papers

Recent Datasets

View All Aristo Datasets

BeliefBank

4998 facts and 12147 constraints to test a model's consistency

Dataset of 4998 simple facts and 12147 constraints to test, and improve, a model's accuracy and consistency

EntailmentBank

2k multi-step entailment trees, explaining the answers to ARC science questions

2k multi-step entailment trees, explaining the answers to ARC science questions

StrategyQA

2,780 implicit multi-hop reasoning questions

StrategyQA is a question-answering benchmark focusing on open-domain questions where the required reasoning steps are implicit in the question and should be inferred using a strategy. StrategyQA includes 2,780 examples, each consisting of a strategy question, its decomposition, and evidence paragraphs.

ProofWriter

Updated RuleTaker datasets with 500k questions, answers and proofs over rulebases.

These datasets accompany the paper "ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language". They contain updated RuleTaker-style datasets with 500k questions, answers and proofs over natural-language rulebases, used to show that Transformers can emulate reasoning over rules expressed in language, including proof generation. It includes variants using closed- and open-world semantics. Proofs include intermediate conclusions. Extra annotations provide data to train the iterative ProofWriter model as well as abductive reasoning to make uncertain statements certain.

“Knowing is not enough, we must apply. Willing is not enough, we must do.”

— Johann Wolfgang von Goethe