Rainbow: A Commonsense Reasoning Benchmark

Mosaic • 2021
Rainbow is a universal commonsense reasoning benchmark spanning both social and physical common sense. Rainbow brings together 6 existing commonsense reasoning tasks: aNLI, Cosmos QA, HellaSWAG, Physical IQa, Social IQa, and WinoGrande. Modelers are challenged to develop techniques which capture world knowledge that helps solve this broad suite of tasks.

Commonsense AI has long been seen as a near impossible goal—until recently. With the advent of large pretrained language models, rich world knowledge has come within reach. Consequently, research interest in common sense has sharply increased with an influx of new benchmarks and models.

Rainbow aims to promote research on common sense models that generalize well over multiple tasks and datasets, bringing together six commonsense reasoning benchmarks that span both the social and the physical: aNLI, Cosmos QA, HellaSWAG, Physical IQa, Social IQa, and WinoGrande.

Our AAAI 2021 paper, Unicorn on Rainbow: A Universal Commonsense Reasoning Model on a New Multitask Benchmark, demonstrates that the tasks within Rainbow transfer well to each other, forming a cohesive whole. See Unicorn on Rainbow to learn more!

Authors

Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi