Allen Institute for AI

AI2 IRVINE

About

The AI2 Irvine office is a collaboration between AI2 scientists and UC Irvine faculty and students, established in May 2018 on the UC Irvine campus. AI2's mission is to contribute to humanity through high-impact AI research and engineering.

AI2 Irvine Team

Our Focus

The focus of AI2 Irvine is fundamental, long-term research on getting machines to read and understand text. This includes carefully defining what it means to read, building models that read, and understanding what models are actually doing when they operate on existing datasets. We have built a large number of linguistically-motivated datasets targeted at pushing the boundaries of machine reading.

Our modeling advances aim for interpretable, compositional reasoning over long sequences of text. With both dataset construction and modeling developments, we strive to ensure that models perform well for the right reasons, using state-of-the-art dataset construction and model analysis techniques, some of which we developed ourselves.

The AI2 Irvine team enjoys a close research collaboration with the University of Califorina, Irvine.

Team

AI2 Irvine Members

  • Matt Gardner's Profile PhotoMatt GardnerResearch
  • Sanjay Subramanian's Profile PhotoSanjay SubramanianPredoctoral Young Investigator

Alumni

  • Jianming Liu's Profile PhotoJianming LiuIntern
  • Eric Wallace's Profile PhotoEric WallaceIntern
  • Yizhong Wang's Profile PhotoYizhong WangIntern
  • Orion Weller's Profile PhotoOrion WellerIntern

UC Irvine Collaborators

  • Sameer Singh's Profile PhotoSameer SinghAssistant Professor
  • Anthony Chen's Profile PhotoAnthony ChenPhD Student
  • Dheeru Dua's Profile PhotoDheeru DuaPhD Student
  • Robert L. Logan IV's Profile PhotoRobert L. Logan IVPhD Student
  • A Framework for Explaining Predictions of NLP Models | AllenNLP, AI2 Irvine

    The AllenNLP Interpret toolkit makes it easy to apply gradient-based saliency maps and adversarial attacks to new models, as well as develop new interpretation methods. AllenNLP Interpret contains three components: a suite of interpretation techniques applicable to most models, APIs for developing new interpretation methods (e.g., APIs to obtain input gradients), and reusable front-end components for visualizing the interpretation results.

    Try the demo
    AllenNLP Interpret text image
  • AllenNLP Interpret text image
    A Framework for Explaining Predictions of NLP Models | AllenNLP, AI2 Irvine

    The AllenNLP Interpret toolkit makes it easy to apply gradient-based saliency maps and adversarial attacks to new models, as well as develop new interpretation methods. AllenNLP Interpret contains three components: a suite of interpretation techniques applicable to most models, APIs for developing new interpretation methods (e.g., APIs to obtain input gradients), and reusable front-end components for visualizing the interpretation results.

    Try the demo
  • AllenNLP demo
    State-of-the-art open source NLP research library | AllenNLP

    AllenNLP is an open source NLP research library that makes it easy for researchers to design and evaluate new deep learning models for nearly any NLP problem, and makes state-of-the-art implementations of several important NLP models and tools readily available for researchers to use and build upon.

    Try the demo
  • AllenNLP demo
    State-of-the-art open source NLP research library | AllenNLP

    AllenNLP is an open source NLP research library that makes it easy for researchers to design and evaluate new deep learning models for nearly any NLP problem, and makes state-of-the-art implementations of several important NLP models and tools readily available for researchers to use and build upon.

    Try the demo
    • Obtaining Faithful Interpretations from Compositional Neural Networks

      Sanjay Subramanian, Ben Bogin, Nitish Gupta, Tomer Wolfson, Sameer Singh, Jonathan Berant, Matt Gardner ACL2020Neural module networks (NMNs) are a popular approach for modeling compositionality: they achieve high accuracy when applied to problems in language and vision, while reflecting the compositional structure of the problem in the network architecture. However, prior work implicitly assumed that the… more
    • QuASE: Question-Answer Driven Sentence Encoding.

      Hangfeng He, Qiang Ning, Dan RothACL2020Question-answering (QA) data often encodes essential information in many facets. This paper studies a natural question: Can we get supervision from QA data for other tasks (typically, non-QA ones)? For example, {\em can we use QAMR (Michael et al., 2017) to improve named entity recognition?} We… more
    • Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering

      Ben Bogin, Sanjay Subramanian, Matt Gardner, Jonathan BerantarXiv2020Answering questions that involve multi-step reasoning requires decomposing them and using the answers of intermediate steps to reach the final answer. However, state-ofthe-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in… more
    • Analyzing Compositionality in Visual Question Answering

      Sanjay Subramanian, Sameer Singh, Matt GardnerNeurIPS • ViGIL Workshop2019Since the release of the original Visual Question Answering (VQA) dataset, several newer datasets for visual reasoning have been introduced, often with the express intent of requiring systems to perform compositional reasoning. Recently, transformer models pretrained on large amounts of images and… more
    • Evaluating Question Answering Evaluation

      Anthony Chen, Gabriel Stanovsky, Sameer Singh, Matt GardnerEMNLP • MRQA Workshop2019As the complexity of question answering (QA) datasets evolve, moving away from restricted formats like span extraction and multiple-choice (MC) to free-form answer generation, it is imperative to understand how well current metrics perform in evaluating QA. This is especially important as existing… more

    Quoref

    24K QA pairs over 4.7K paragraphs, split between train (19K QAs), development (2.4K QAs) and a hidden test partition (2.5K QAs).

    Quoref is a QA dataset which tests the coreferential reasoning capability of reading comprehension systems. In this span-selection benchmark containing 24K questions over 4.7K paragraphs from Wikipedia, a system must resolve hard coreferences before selecting the appropriate span(s) in the paragraphs for answering questions.

    ROPES

    14k QA pairs over 1.7K paragraphs, split between train (10k QAs), development (1.6k QAs) and a hidden test partition (1.7k QAs).

    ROPES is a QA dataset which tests a system's ability to apply knowledge from a passage of text to a new situation. A system is presented a background passage containing a causal or qualitative relation(s), a novel situation that uses this background, and questions that require reasoning about effects of the relationships in the back-ground passage in the context of the situation.

    DROP

    The DROP dataset contains 96k QA pairs over 6.7K paragraphs, split between train (77k QAs), development (9.5k QAs) and a hidden test partition (9.5k QAs).

    DROP is a QA dataset that tests the comprehensive understanding of paragraphs. In this crowdsourced, adversarially-created, 96k question-answering benchmark, a system must resolve multiple references in a question, map them onto a paragraph, and perform discrete operations over them (such as addition, counting, or sorting).

    ICS Partnership with AI2 Leads to a New Toolkit and Best Demo Paper Award

    UCI Dept. of Computer Science
    November 19, 2019
    Read the Article

    AI/NLP Research Partnership with Allen Institute for AI

    UCI CML
    September 30, 2019
    Read the Article

    ICS Partnership with Allen Institute for AI Advances Machine Learning

    UCI Dept. of Computer Science
    April 24, 2019
    Read the Article