Papers

Learn more about AI2's Lasting Impact Award
Viewing 21-30 of 145 papers
  • Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study

    Rahul Nadkarni, David Wadden, Iz Beltagy, Noah A. Smith, Hannaneh Hajishirzi, Tom HopeAKBC2021 Biomedical knowledge graphs (KGs) hold rich information on entities such as diseases, drugs, and genes. Predicting missing links in these graphs can boost many important applications, such as drug design and repurposing. Recent work has shown that general…
  • Expected Validation Performance and Estimation of a Random Variable's Maximum

    Jesse Dodge, Suchin Gururangan, D. Card, Roy Schwartz, Noah A. SmithFindings of EMNLP2021 Research in NLP is often supported by experimental results, and improved reporting of such results can lead to better understanding and more reproducible science. In this paper we analyze three statistical estimators for expected validation performance, a…
  • Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation

    Jungo Kasai, Nikolaos Pappas, Hao Peng, James Cross, Noah A. SmithICLR2021 State-of-the-art neural machine translation models generate outputs autoregressively, where every step conditions on the previously generated tokens. This sequential nature causes inherent decoding latency. Non-autoregressive translation techniques, on the…
  • Random Feature Attention

    Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, Lingpeng KongICLR2021 Transformers are state-of-the-art models for a variety of sequence modeling tasks. At their core is an attention function which models pairwise interactions between the inputs at every timestep. While attention is powerful, it does not scale efficiently to…
  • All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text

    Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, Noah A. SmithACL2021 Human evaluations are typically considered the gold standard in natural language generation, but as models' fluency improves, how well can evaluators detect and judge machine-generated text? We run a study assessing non-experts' ability to distinguish between…
  • DExperts: On-the-Fly Controlled Text Generation with Experts and Anti-Experts

    Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi ACL2021 Despite recent advances in natural language generation, it remains challenging to control attributes of generated text. We propose DEXPERTS: Decoding-time Experts, a decodingtime method for controlled text generation that combines a pretrained language model…
  • Effective Attention Sheds Light On Interpretability

    Kaiser Sun and Ana MarasovićFindings of ACL2021 An attention matrix of a transformer selfattention sublayer can provably be decomposed into two components and only one of them (effective attention) contributes to the model output. This leads us to ask whether visualizing effective attention gives different…
  • Explaining NLP Models via Minimal Contrastive Editing (MiCE)

    Alexis Ross, Ana Marasović, Matthew E. PetersFindings of ACL2021 Humans give contrastive explanations that explain why an observed event happened rather than some other counterfactual event (the contrast case). Despite the important role that contrastivity plays in how people generate and evaluate explanations, this…
  • Explaining Relationships Between Scientific Documents

    Kelvin Luu, Xinyi Wu, Rik Koncel-Kedziorski, Kyle Lo, Isabel Cachola, Noah A. SmitACL2021 We address the task of explaining relationships between two scientific documents using natural language text. This task requires modeling the complex content of long technical documents, deducing a relationship between these documents, and expressing that…
  • PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World

    Rowan Zellers, Ari Holtzman, Matthew E. Peters, R. Mottaghi, Aniruddha Kembhavi, A. Farhadi, Yejin ChoiACL2021 We propose PIGLeT: a model that learns physical commonsense knowledge through interaction, and then uses this knowledge to ground language. We factorize PIGLeT into a physical dynamics model, and a separate language model. Our dynamics model learns not just…