As AI systems become an increasing part of people’s everyday lives, it becomes increasingly important that they understand people’s ethical norms. Norm understanding involves both prescriptive ethics (how should one act?) and descriptive ethics (what do people believe?).
Scruples facilitates research on descriptive machine ethics, or machines’ ability to understand people’s ethical judgments. Scruples builds upon real-world examples of ethical situations. Similarly, it offers many annotations for each example in order to capture the inherent diversity of ethical judgments often found in communities.
Scruples divides into two parts: the Anecdotes and the Dilemmas. The Anecdotes collect more than 625,000 ethical judgments over 32,000 real-life anecdotes. Each recounts a complex ethical situation, often posing moral dilemmas, and comes with a distribution of judgments contributed by community members. In contrast, the Dilemmas present pairs of simpler actions ranked in terms of which is less ethical by crowdworkers.
See our AAAI 2021 paper: Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-life Anecdotes for more information on the datasets, baselines, and new methodological tools for tackling Scruples.
Details | Created | Cross Entropy |
---|---|---|
1 BERT + Dirichlet-Multinomial Nicholas Lourie, Ronan Le Bras, Yejin Choi | 8/27/2022 | 106% |
2 RoBERTa + Dirichlet-Multinomial Nicholas Lourie, Ronan Le Bras, Yejin Choi | 8/27/2022 | 103% |
Nicholas Lourie, Ronan Le Bras, Yejin Choi