ARISTO

Build machines that read, learn and reason.

The Aristo Project aims to build systems that demonstrate a deep understanding of the world, integrating technologies for reading, learning, reasoning, and explanation.

A multiple choice question and reasoning explaining each answer

Our research integrates multiple AI technologies, including:

Natural language processing
Information extraction
Knowledge representation
Machine reasoning
Commonsense knowledge

Research Areas

Probing Reasoning with Language Models

Language models (LMs) have dominated much of AI recently. But what kind(s) of reasoning are they capable of? And how can they be taught to do more? We are developing analytical datasets to probe LMs and help answer these questions.

Learn More:

Probing Natural Language Inference Models through Semantic Fragments

Multihop Reasoning

Many questions require multiple pieces of information to be combined to arrive at an answer. We are developing new multihop models capable of identifying and combining relevant facts to answer such questions.

Learn More:

Repurposing Entailment for Multi-Hop Question Answering Tasks

QASC: A Dataset for Question Answering via Sentence Composition

Explanation

An intelligent system should not only answer questions correctly, but also be able to explain why its answers are correct. Such a capability is essential for practical acceptance of AI technology. It is also essential for the broader goals of communicating knowledge to a user, and receiving correction from the user when the system's answer is wrong.

Learn More:

What's Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering

Exploiting Explicit Paths for Multi-hop Reading Comprehension

Reasoning about Actions

A key aspect of intelligence is being able to reason about the dynamics of the world. This requires modeling what state the world might be in, and how different actions might affect that state. Such capabilities are essential for understanding what happens during a procedure or process, for planning, and for reasoning about "what if..." scenarios.

Learn More:

Reasoning about Actions and State Changes by Injecting Commonsense Knowledge

Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text

Demos

View All Aristo Demos

RuleTaker

Transformers as Soft Reasoners over Language | Aristo

RuleTaker determines whether statements are True or False based on rules given in natural language.

Try the demo

RuleTaker

Transformers as Soft Reasoners over Language | Aristo

RuleTaker determines whether statements are True or False based on rules given in natural language.

Try the demo

UnifiedQA

Crossing format boundaries with a single QA system | Aristo

UnifiedQA is a single pre-trained QA model that performs surprisingly well across 17 QA datasets spanning 4 diverse formats. Fine-tuning UnifiedQA into specialized models results in a new state-of-the-art on 6 datasets, establishing this model as a strong starting point for building QA systems.

Try the demo

UnifiedQA

Crossing format boundaries with a single QA system | Aristo

Try the demo

Recent Papers

View All Aristo Papers

Everything Happens for a Reason: Discovering the Purpose of Actions in Procedural Text
Bhavana Dalvi Mishra, Niket Tandon, Antoine Bosselut, Wen-tau Yih, Peter ClarkEMNLP • 2019Our goal is to better comprehend procedural text, e.g., a paragraph about photosynthesis, by not only predicting what happens, but why some actions need to happen before others. Our approach builds on a prior process comprehension framework for predicting actions' effects, to also identify… more
QASC: A Dataset for Question Answering via Sentence Composition
Tushar Khot, Peter Clark, Michal Guerquin, Paul Edward Jansen, Ashish Sabharwal AAAI • 2020Composing knowledge from multiple pieces of texts is a key challenge in multi-hop question answering. We present a multi-hop reasoning dataset, Question Answering via Sentence Composition (QASC), that requires retrieving facts from a large corpus and composing them to answer a multiple-choice… more
Probing Natural Language Inference Models through Semantic Fragments
Kyle Richardson, Hai Na Hu, Lawrence S. Moss, Ashish SabharwalAAAI • 2020Do state-of-the-art models for language understanding already have, or can they easily learn, abilities such as boolean coordination, quantification, conditionals, comparatives, and monotonicity reasoning (i.e., reasoning about word substitutions in sentential contexts)? While such phenomena are… more
CLUE: A Chinese Language Understanding Evaluation Benchmark
L. Xu, X.Zhang, L. Li, H. Hu, C. Cao, W. Liu, J. Li, Y. Li, K. Sun, Y. Xu, Y. Cui, C. Yu, Q. Dong, Y. Tian, D. Yu, B. Shi, J. Zeng, R. Wang, W. Xie, Y. Li, Y. Patterson, Z. Tian, Y. Zhang, H. Zhou, S. Liu, Q. Zhao, C. Yue, X. Zhang, Z. Yang, et.al.COLING 2020 • 2020We introduce CLUE, a Chinese Language Understanding Evaluation benchmark. It contains eight different tasks, including single-sentence classification, sentence pair classification, and machine reading comprehension. We evaluate CLUE on a number of existing full-network pre-trained models for… more
Belief Propagation Neural Networks
J. Kuck, Shuvam Chakraborty, Hao Tang, R. Luo, Jiaming Song, A. Sabharwal, S. ErmonNeurIPS • 2020Learned neural solvers have successfully been used to solve combinatorial optimization and decision problems. More general counting variants of these problems, however, are still largely solved with hand-crafted solvers. To bridge this gap, we introduce belief propagation neural networks (BPNNs), a… more

View All Aristo Papers

Recent Datasets

View All Aristo Datasets

QuaRTz Dataset

3864 questions about open domain qualitative relationships

QuaRTz is a crowdsourced dataset of 3864 multiple-choice questions about open domain qualitative relationships. Each question is paired with one of 405 different background sentences (sometimes short paragraphs).

QuaRel Dataset

2771 story questions about qualitative relationships

QuaRel is a crowdsourced dataset of 2771 multiple-choice story questions, including their logical forms.

Open PI dataset

A Dataset for Tracking Entities in Open Domain Procedural Text

We present the first dataset for tracking state changes in procedural text from arbitrary do-mains by using an unrestricted (open) vocabulary. For example, in a text describing fog removal using potatoes, a car window may transition between being foggy, sticky, opaque, and clear. Previous formulations of this task provide the text and entities involved, and ask how those entities change for just a small, pre-defined set of attributes (e.g., location), limiting their fidelity. Our solution is a new task formulation in which just the text is provided, from which a set of state changes (entity, at-tribute, before, after) is generated for each step, where the entity, attribute, and values must all be predicted from an open vocabulary. Using crowdsourcing, we create OPENPI, a high-quality (vetted by humans), and large-scale dataset comprising 33K state changes over 4,050 sentences from 810 procedural real-world paragraphs from WikiHow.com. An state-of-the-art generation model on this task achieves 18% F1 based on BLEU, leaving a lot of room for novel model architectures.

eQASC: Multihop Explanations for QASC

98k annotated explanations for the QASC dataset

This dataset contains 98k 2-hop explanations for questions in the QASC dataset, with annotations indicating if they are valid (~25k) or invalid (~73k) explanations.

“Knowing is not enough, we must apply. Willing is not enough, we must do.”

— Johann Wolfgang von Goethe