Papers
See AI2's Award Winning Papers
Learn more about AI2's Lasting Impact Award
Viewing 11-20 of 145 papers
Generative Context Pair Selection for Multi-hop Question Answering
Dheeru Dua, Cicero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer SinghEMNLP • 2021 Compositional reasoning tasks like multi-hop question answering, require making latent decisions to get the final answer, given a question. However, crowdsourced datasets often capture only a slice of the underlying task distribution, which can induce…Learning with Instance Bundles for Reading Comprehension
Dheeru Dua, Pradeep Dasigi, Sameer Singh and Matt GardnerEMNLP • 2021 When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these…Measuring Association Between Labels and Free-Text Rationales
Sarah Wiegreffe, Ana Marasović, Noah A. SmithEMNLP • 2021 Interpretable NLP has taking increasing interest in ensuring that explanations are faithful to the model’s decision-making process. This property is crucial for machine learning researchers and practitioners using explanations to better understand models…Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization
Ansong Ni, Matt Gardner, Pradeep DasigiEMNLP • 2021 Question Answering (QA) tasks requiring information from multiple documents often rely on a retrieval model to identify relevant information from which the reasoning model can derive an answer. The retrieval model is typically trained to maximize the…Paired Examples as Indirect Supervision in Latent Decision Models
Nitish Gupta, Sameer Singh, Matt Gardner and Dan RothEMNLP • 2021 Compositional, structured models are appealing because they explicitly decompose problems and provide interpretable intermediate outputs that give confidence that the model is not simply latching onto data artifacts. Learning these models is challenging…Parameter Norm Growth During Training of Transformers
William Merrill, Vivek Ramanujan, Yoav Goldberg, Roy Schwartz, Noah A. Smith EMNLP • 2021 The capacity of neural networks like the widely adopted transformer is known to be very high. Evidence is emerging that they learn successfully due to inductive bias in the training routine, typically some variant of gradient descent (GD). To better…Probing Across Time: What Does RoBERTa Know and When?
Leo Z. Liu, Yizhong Wang, Jungo Kasai, Hanna Hajishirzi, Noah A. SmithFindings of EMNLP • 2021 Models of language trained on very large corpora have been demonstrated useful for NLP. As fixed artifacts, they have become the object of intense study, with many researchers “probing” the extent to which linguistic abstractions, factual and commonsense…Sentence Bottleneck Autoencoders from Transformer Language Models
Ivan Montero, Nikolaos Pappas, Noah A. SmithEMNLP • 2021 Representation learning for text via pretraining a language model on a large corpus has become a standard starting point for building NLP systems. This approach stands in contrast to autoencoders, also trained on raw text, but with the objective of learning…Understanding Mention Detector-Linker Interaction in Neural Coreference Resolution
Zhaofeng Wu, Matt GardnerEMNLP • CRAC • 2021 Despite significant recent progress in coreference resolution, the quality of current state-of-the-art systems still considerably trails behind human-level performance. Using the CoNLL-2012 and PreCo datasets, we dissect the best instantiation of the…DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization
Zeqiu Wu, Bo-Ru Lu, Hannaneh Hajishirzi, Mari OstendorfEMNLP • 2021 Identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation. We introduce a knowledge identification model that leverages the document structure to provide dialogue…