Papers

Learn more about AI2's Lasting Impact Award
Viewing 11-20 of 145 papers
  • Generative Context Pair Selection for Multi-hop Question Answering

    Dheeru Dua, Cicero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer SinghEMNLP2021 Compositional reasoning tasks like multi-hop question answering, require making latent decisions to get the final answer, given a question. However, crowdsourced datasets often capture only a slice of the underlying task distribution, which can induce…
  • Learning with Instance Bundles for Reading Comprehension

    Dheeru Dua, Pradeep Dasigi, Sameer Singh and Matt GardnerEMNLP2021 When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these…
  • Measuring Association Between Labels and Free-Text Rationales

    Sarah Wiegreffe, Ana Marasović, Noah A. SmithEMNLP2021 Interpretable NLP has taking increasing interest in ensuring that explanations are faithful to the model’s decision-making process. This property is crucial for machine learning researchers and practitioners using explanations to better understand models…
  • Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization

    Ansong Ni, Matt Gardner, Pradeep DasigiEMNLP2021 Question Answering (QA) tasks requiring information from multiple documents often rely on a retrieval model to identify relevant information from which the reasoning model can derive an answer. The retrieval model is typically trained to maximize the…
  • Paired Examples as Indirect Supervision in Latent Decision Models

    Nitish Gupta, Sameer Singh, Matt Gardner and Dan RothEMNLP2021 Compositional, structured models are appealing because they explicitly decompose problems and provide interpretable intermediate outputs that give confidence that the model is not simply latching onto data artifacts. Learning these models is challenging…
  • Parameter Norm Growth During Training of Transformers

    William Merrill, Vivek Ramanujan, Yoav Goldberg, Roy Schwartz, Noah A. Smith EMNLP2021 The capacity of neural networks like the widely adopted transformer is known to be very high. Evidence is emerging that they learn successfully due to inductive bias in the training routine, typically some variant of gradient descent (GD). To better…
  • Probing Across Time: What Does RoBERTa Know and When?

    Leo Z. Liu, Yizhong Wang, Jungo Kasai, Hanna Hajishirzi, Noah A. SmithFindings of EMNLP2021 Models of language trained on very large corpora have been demonstrated useful for NLP. As fixed artifacts, they have become the object of intense study, with many researchers “probing” the extent to which linguistic abstractions, factual and commonsense…
  • Sentence Bottleneck Autoencoders from Transformer Language Models

    Ivan Montero, Nikolaos Pappas, Noah A. SmithEMNLP2021 Representation learning for text via pretraining a language model on a large corpus has become a standard starting point for building NLP systems. This approach stands in contrast to autoencoders, also trained on raw text, but with the objective of learning…
  • Understanding Mention Detector-Linker Interaction in Neural Coreference Resolution

    Zhaofeng Wu, Matt GardnerEMNLP • CRAC2021 Despite significant recent progress in coreference resolution, the quality of current state-of-the-art systems still considerably trails behind human-level performance. Using the CoNLL-2012 and PreCo datasets, we dissect the best instantiation of the…
  • DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization

    Zeqiu Wu, Bo-Ru Lu, Hannaneh Hajishirzi, Mari OstendorfEMNLP2021 Identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation. We introduce a knowledge identification model that leverages the document structure to provide dialogue…