Papers

Learn more about AI2's Lasting Impact Award
Viewing 21-30 of 180 papers
  • Log-Precision Transformers are Uniform Threshold Circuits

    William Merrill, Ashish SabharwalarXiv2022 We prove that transformer neural networks with logarithmic precision in the input length (and where the feedforward subnetworks are computable using linear space in their input length) can be simulated by constant-depth uniform threshold circuits. Thus, such…
  • DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Models

    Gregor Betz, Kyle RichardsonSEM2022 In this paper, we present and implement a multi-dimensional, modular framework for performing deep argument analysis (DeepA2) using current pre-trained language models (PTLMs). ArgumentAnalyst – a T5 model (Raffel et al. 2020) set up and trained within DeepA2…
  • Retrieval Data Augmentation Informed by Downstream Question Answering Performance

    James Ferguson, Pradeep Dasigi, Tushar Khot, Hannaneh HajishirziACL • FEVER2022 Training retrieval models to fetch contexts for Question Answering (QA) over large corpora requires labeling relevant passages in those corpora. Since obtaining exhaustive manual annotations of all relevant passages is not feasible, prior work uses text…
  • Cross-Task Generalization via Natural Language Crowdsourcing Instructions

    Swaroop Mishra, Daniel Khashabi, Chitta Baral, Hanna HajishirziACL2022 Can we enable NLP models to appropriately respond to instructional prompts and consequently generalize to new tasks? To study this question, we leverage the existing NLP datasets and the instructions that were used to crowdsource them to create…
  • Hey AI, Can You Solve Complex Tasks by Talking to Agents?

    Tushar Khot, Kyle Richardson, Daniel Khashabi, Ashish SabharwalFindings of ACL2022 Humans often solve complex problems by interacting (in natural language) with existing agents, such as AI assistants, that can solve simpler sub-tasks. These agents themselves can be powerful systems built using extensive resources and privately held data. In…
  • NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks

    Swaroop Mishra, Arindam Mitra, Neeraj Varshney, Bhavdeep Singh Sachdeva, Peter Clark, Chitta Baral, A. KalyanACL2022 Given the ubiquitous nature of numbers in text, reasoning with numbers to perform simple calculations is an important skill of AI systems. While many datasets and models have been developed to this end, state-of-the-art AI systems are brittle; failing to…
  • Better Retrieval May Not Lead to Better Question Answering

    Zhengzhong Liang, Tushar Khot, Steven Bethard, Mihai Surdeanu, Ashish SabharwalarXiv2022 Considerable progress has been made recently in open-domain question answering (QA) problems, which require Information Retrieval (IR) and Reading Comprehension (RC). A popular approach to improve the system's performance is to improve the quality of the…
  • Saturated Transformers are Constant-Depth Threshold Circuits

    William Merrill, Ashish Sabharwal, Noah A. SmithTACL2022 Transformers have become a standard neural network architecture for many NLP problems, motivating theoretical analysis of their power in terms of formal languages. Recent work has shown that transformers with *hard* attention are quite limited in power, as…
  • Towards Teachable Reasoning Systems

    Bhavana Dalvi, Oyvind Tafjord, Peter ClarkarXiv2022 Our goal is a teachable reasoning system for question-answering (QA), where a user can interact with faithful answer explanations, and correct errors so that the system improves over time. Our approach is three-fold: First, generated chains of reasoning show…
  • Memory-assisted prompt editing to improve GPT-3 after deployment

    Aman Madaan, Niket Tandon, Peter Clark, Yiming YangACL • Workshop on Commonsense Reasoning2022 Large LMs such as GPT-3 are powerful, but can commit mistakes that are obvious to humans. For example, GPT-3 would mistakenly interpret "What word is similar to good?" to mean a homonym, while the user intended a synonym. Our goal is to effectively correct…