Papers
See AI2's Award Winning Papers
Learn more about AI2's Lasting Impact Award
Viewing 1-10 of 844 papers
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors
Hamish Ivison, Noah A. Smith, Hannaneh Hajishirzi, Pradeep DasigiACL Findings • 2023 Language models trained on massive prompted multitask datasets like T0 (Sanh et al., 2021) or FLAN (Wei et al., 2021a) can generalize to tasks unseen during training. We show that training on a carefully chosen subset of instances can outperform training on…HINT: Hypernetwork Instruction Tuning for Efficient Few- and Zero-Shot Generalisation
Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew E. PetersACL • 2023 Recent NLP models have shown the remarkable ability to effectively generalise `zero-shot' to new tasks using only natural language instructions as guidance. However, many of these approaches suffer from high computational costs due to their reliance on…Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation
Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, D. Klakow, Yanai ElazarFindings of ACL 2023 • 2023 Few-shot fine-tuning and in-context learning are two alternative strategies for task adaptation of pre-trained language models. Recently, in-context learning has gained popularity over fine-tuning due to its simplicity and improved out-of-domain…Improving the reliability of ML-corrected climate models with novelty detection.
Clayton Sanford, Anna Kwa, Oliver Watt-Meyer, Spencer K. Clark, Noah D. Brenowitz, Jeremy McGibbon, and Christopher S. BrethertonESSOAr • 2023 The use of machine learning (ML) for the online correction of coarse-resolution atmospheric models has proven effective in reducing biases in near-surface temperature and precipitation rate. However, this often introduces biases in the upper atmosphere and…Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback
Yao Fu, Hao-Chun Peng, Tushar Khot, Mirella LapataarXiv.org • 2023 We study whether multiple large language models (LLMs) can autonomously improve each other in a negotiation game by playing, reflecting, and criticizing. We are interested in this question because if LLMs were able to improve each other, it would imply the…Pace v0.2: a Python-based performance-portable atmospheric model
Johann Dahm, Eddie Davis, Florian Deconinck, Oliver Elbert, Rhea George, Jeremy McGibbon, Tobias Wicky,Elynn Wu,Christopher Kung, Tal Ben-Nun, Lucas Harris , Linus Groner, Oliver FuhrerGeoscientific Model Development • 2023 Progress in leveraging current and emerging high-performance computing infrastructures using traditional weather and climate models has been slow. This has become known more broadly as the software productivity gap. With the end of Moore's law driving forward…From Centralized to Ad-Hoc Knowledge Base Construction for Hypotheses Generation.
Shaked Launer-Wachs, Hillel Taub-Tabib, Jennie Tokarev Madem, Orr Bar-Natan, Yoav Goldberg, Y. ShamayJournal of Biomedical Informatics • 2023 Objective To demonstrate and develop an approach enabling individual researchers or small teams to create their own ad-hoc, lightweight knowledge bases tailored for specialized scientific interests, using text-mining over scientific literature, and…Machine-Learned Climate Model Corrections From a Global Storm-Resolving Model: Performance Across the Annual Cycle
Anna Kwa, Spencer K. Clark, Brian Henn, Noah D. Brenowitz, Jeremy McGibbon, Oliver Watt-Meyer, W. Andre Perkins, Lucas Harris, Christopher S. BrethertonJournal of Advances in Modeling Earth Systems • 2023 One approach to improving the accuracy of a coarse-grid global climate model is to add machine-learned (ML) state-dependent corrections to the prognosed model tendencies, such that the climate model evolves more like a reference fine-grid global storm…Embedding Recycling for Language Models
Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug DowneyFindings of EACL • 2023 Training and inference with large neural models is expensive. However, for many application domains, while new tasks and models arise frequently, the underlying doc-uments being modeled remain mostly un-changed. We study how to decrease computational cost in…Complexity-Based Prompting for Multi-Step Reasoning
Yao Fu, Hao-Chun Peng, Ashish Sabharwal, Peter Clark, Tushar KhotICLR • 2023 We study the task of prompting large-scale language models to perform multi-step reasoning. Existing work shows that when prompted with a chain of thoughts (CoT), sequences of short sentences describing intermediate reasoning steps towards a final answer…