About AllenNLP

The AllenNLP team envisions language-centered AI that equitably serves humanity. We work to improve NLP systems' performance and accountability, and advance scientific methodologies for evaluating and understanding those systems. We deliver high-impact research of our own and masterfully-engineered open-source tools to accelerate NLP research around the world.

Featured Software

AI2 Tango

A Python library for choreographing your machine learning research. Construct machine learning experiments out of repeatable, reusable steps.

View

AllenNLP Library

A natural language processing platform for building state-of-the-art models. A complete platform for solving natural language processing tasks in PyTorch.

View
  • Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks

    Akari Asai, Matt Gardner, Hannaneh HajishirziNAACL2022 Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open-domain question answering and fact verification. These models are trained to generate a final output given retrieved passages…
  • FaVIQ: FAct Verification from Information-seeking Questions

    Jungsoo Park, Sewon Min, Jaewoo Kang, Luke Zettlemoyer, Hannaneh HajishirziACL2022 Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic real-world claims. Existing claims are either authored by crowdworkers, thereby introducing…
  • MetaICL: Learning to Learn In Context

    Sewon Min, M. Lewis, Luke Zettlemoyer, Hannaneh HajishirziNAACL2022 We introduce MetaICL (Meta-training for In-Context Learning), a new meta-training framework for few-shot learning where a pretrained language model is tuned to do in-context learning on a large set of training tasks. This meta-training enables the model to…
  • MultiVerS: Improving scientific claim verification with weak supervision and full-document context

    David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh HajishirziNAACL2022 The scientific claim verification task requires an NLP system to label scientific documents which S UPPORT or R EFUTE an input claim, and to select evidentiary sentences (or rationales ) justifying each predicted label. In this work, we present M ULTI V ER S…
  • Noisy Channel Language Model Prompting for Few-Shot Text Classification

    Sewon Min, Michael Lewis, Hannaneh Hajishirzi, Luke ZettlemoyerACL2022 We introduce a noisy channel approach for language model prompting in few-shot text classification. Instead of computing the likelihood of the label given the input (referred as direct models), channel models compute the conditional probability of the input…

Qasper

Question Answering on Research Papers

A dataset containing 1585 papers with 5049 information-seeking questions asked by regular readers of NLP papers, and answered by a separate set of NLP practitioners.

A Dataset of Incomplete Information Reading Comprehension Questions

13K reading comprehension questions on Wikipedia paragraphs that require following links in those paragraphs to other Wikipedia pages

IIRC is a crowdsourced dataset consisting of information-seeking questions requiring models to identify and then retrieve necessary information that is missing from the original context. Each original context is a paragraph from English Wikipedia and it comes with a set of links to other Wikipedia pages, and answering the questions requires finding the appropriate links to follow and retrieving relevant information from those linked pages that is missing from the original context.

ZEST: ZEroShot learning from Task descriptions

ZEST is a benchmark for zero-shot generalization to unseen NLP tasks, with 25K labeled instances across 1,251 different tasks.

ZEST tests whether NLP systems can perform unseen tasks in a zero-shot way, given a natural language description of the task. It is an instantiation of our proposed framework "learning from task descriptions". The tasks include classification, typed entity extraction and relationship extraction, and each task is paired with 20 different annotated (input, output) examples. ZEST's structure allows us to systematically test whether models can generalize in five different ways.

MOCHA

A benchmark for training and evaluating generative reading comprehension metrics.

Posing reading comprehension as a generation problem provides a great deal of flexibility, allowing for open-ended questions with few restrictions on possible answers. However, progress is impeded by existing generation metrics, which rely on token overlap and are agnostic to the nuances of reading comprehension. To address this, we introduce a benchmark for training and evaluating generative reading comprehension metrics: MOdeling Correctness with Human Annotations. MOCHA contains 40K human judgement scores on model outputs from 6 diverse question answering datasets and an additional set of minimal pairs for evaluation. Using MOCHA, we train an evaluation metric: LERC, a Learned Evaluation metric for Reading Comprehension, to mimic human judgement scores.

How to shrink AI’s ballooning carbon footprint

Nature
July 19, 2022
Read the Article

These simple changes can make AI research much more energy efficient

MIT Tech Review
July 6, 2022
Read the Article

Measuring AI’s Carbon Footprint

IEEE Spectrum
June 26, 2022
Read the Article

Why Historical Language Is a Challenge for Artificial Intelligence

unite.ai
November 16, 2021
Read the Article

The curse of neural toxicity: AI2 and UW researchers help computers watch their language

GeekWire
March 6, 2021
Read the Article

Green AI

CACM
November 18, 2020
Read the Article

Your favorite A.I. language tool is toxic

Fortune
September 29, 2020
Read the Article

Deep Learning’s Climate Change Problem

Forbes
June 17, 2020
Read the Article

Podcasts

  • NLP Highlights

    NLP Highlights is AllenNLP’s podcast for discussing recent and interesting work related to natural language processing. Hosts from the AllenNLP team at AI2 offer short discussions of papers and occasionally interview authors about their work.

    You can also find NLP Highlights on Apple Podcasts, Spotify, PlayerFM, or Stitcher.