Skip to main content ->
Ai2

Research - Papers

Explore a selection of our published work on a variety of key research challenges in AI.

Filter papers

CORD-19: The Covid-19 Open Research Dataset

L. Lu WangK. LoY. ChandrasekharS. Kohlmeier
2020
ACL • NLP-COVID

The Covid-19 Open Research Dataset (CORD-19) is a growing 1 resource of scientific papers on Covid-19 and related historical coronavirus research. CORD-19 is designed to facilitate the development… 

SUPP. AI: finding evidence for supplement-drug interactions

Lucy Lu WangOyvind TafjordArman CohanWaleed Ammar
2020
ACL• Demo

Dietary supplements are used by a large portion of the population, but information on their pharmacologic interactions is incomplete. To address this challenge, we present this http URL, an… 

A Formal Hierarchy of RNN Architectures

William. MerrillGail Garfinkel WeissYoav GoldbergEran Yahav
2020
ACL

We develop a formal hierarchy of the expressive capacity of RNN architectures. The hierarchy is based on two formal properties: space complexity, which measures the RNN's memory, and rational… 

A Mixture of h-1 Heads is Better than h Heads

Hao PengRoy SchwartzDianqi LiNoah A. Smith
2020
ACL

Multi-head attentive neural architectures have achieved state-of-the-art results on a variety of natural language processing tasks. Evidence has shown that they are overparameterized; attention… 

A Two-Stage Masked LM Method for Term Set Expansion

Guy KushilevitzShaul MarkovitchYoav Goldberg
2020
ACL

We tackle the task of Term Set Expansion (TSE): given a small seed set of example terms from a semantic class, finding more members of that class. The task is of great practical utility, and also of… 

Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

Suchin GururanganAna MarasovićSwabha SwayamdiptaNoah A. Smith
2020
ACL

Language models pretrained on text from a wide variety of sources form the foundation of today's NLP. In light of the success of these broad-coverage models, we investigate whether it is still… 

Improving Transformer Models by Reordering their Sublayers

Ofir PressNoah A. SmithOmer Levy
2020
ACL

Multilayer transformer networks consist of interleaved self-attention and feedforward sublayers. Could ordering the sublayers in a different pattern lead to better performance? We generate randomly… 

Injecting Numerical Reasoning Skills into Language Models

Mor GevaAnkit GuptaJonathan Berant
2020
ACL

Large pre-trained language models (LMs) are known to encode substantial amounts of linguistic information. However, high-level reasoning skills, such as numerical reasoning, are difficult to learn… 

Interactive Extractive Search over Biomedical Corpora

Hillel Taub-TabibMicah ShlainShoval SaddeYoav Goldberg
2020
ACL

We present a system that allows life-science researchers to search a linguistically annotated corpus of scientific texts using patterns over dependency graphs, as well as using patterns over token… 

Language (Re)modelling: Towards Embodied Language Understanding

Ronen TamariChen ShaniTom HopeDafna Shahaf
2020
ACL

While natural language understanding (NLU) is advancing rapidly, today’s technology differs from human-like language understanding in fundamental ways, notably in its inferior efficiency,…