Research - Papers
Explore a selection of our published work on a variety of key research challenges in AI.
Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?
With the growing popularity of deep-learning based NLP models, comes a need for interpretable systems. But what is interpretability, and what constitutes a high-quality interpretation? In this…
Unsupervised Domain Clusters in Pretrained Language Models
The notion of "in-domain data" in NLP is often over-simplistic and vague, as textual data varies in many nuanced linguistic aspects such as topic, style or level of formality. In addition, domain…
Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering
Answering questions that involve multi-step reasoning requires decomposing them and using the answers of intermediate steps to reach the final answer. However, state-ofthe-art models in grounded…
When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM Predictions
A growing body of work makes use of probing in order to investigate the working of neural models, often considered black boxes. Recently, an ongoing debate emerged surrounding the limitations of the…
Evaluating NLP Models via Contrast Sets
Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading:…
Differentiable Scene Graphs
Understanding the semantics of complex visual scenes involves perception of entities and reasoning about their relations. Scene graphs provide a natural representation for these tasks, by assigning…
On Making Reading Comprehension More Comprehensive
Machine reading comprehension, the task of evaluating a machine’s ability to comprehend a passage of text, has seen a surge in popularity in recent years. There are many datasets that are targeted…
ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine Reading Comprehension
Reading comprehension is one of the crucial tasks for furthering research in natural language understanding. A lot of diverse reading comprehension datasets have recently been introduced to study…
Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing
A major hurdle on the road to conversational interfaces is the difficulty in collecting data that maps language utterances to logical forms. One prominent approach for data collection has been to…
Global Reasoning over Database Structures for Text-to-SQL Parsing
State-of-the-art semantic parsers rely on auto-regressive decoding, emitting one symbol at a time. When tested against complex databases that are unobserved at training time (zero-shot), the parser…