Research - Papers
Explore a selection of our published work on a variety of key research challenges in AI.
Segment-Phrase Table for Semantic Segmentation, Visual Entailment and Paraphrasing
We introduce Segment-Phrase Table (SPT), a large collection of bijective associations between textual phrases and their corresponding segmentations. Leveraging recent progress in object recognition…
Solving Geometry Problems: Combining Text and Diagram Interpretation
This paper introduces GeoS, the first automated system to solve unaltered SAT geometry questions by combining text understanding and diagram interpretation. We model the problem of understanding…
Discriminative and Consistent Similarities in Instance-Level Multiple Instance Learning
In this paper we present a bottom-up method to instance level Multiple Instance Learning (MIL) that learns to discover positive instances with globally constrained reasoning about local pairwise…
Generating Notifications for Missing Actions: Don’t forget to turn the lights off!
We all have experienced forgetting habitual actions among our daily activities. For example, we probably have forgotten to turn the lights off before leaving a room or turn the stove off after…
Looking Beyond Text: Extracting Figures, Tables and Captions from Computer Science Papers
Identifying and extracting figures and tables along with their captions from scholarly articles is important both as a way of providing tools for article summarization, and as part of larger systems…
VISALOGY: Answering Visual Analogy Questions
In this paper, we study the problem of answering visual analogy questions. These questions take the form of image A is to image B as image C is to what. Answering these questions entails discovering…
VisKE: Visual Knowledge Extraction and Question Answering by Visual Verification of Relation Phrases
How can we know whether a statement about our world is valid. For example, given a relationship between a pair of entities e.g., 'eat(horse, hay)', how can we know whether this relationship is true…
Learning Everything about Anything: Webly-Supervised Visual Concept Learning
Recognition is graduating from labs to real-world applications. While it is encouraging to see its potential being tapped, it brings forth a fundamental challenge to the vision researcher:…
Diagram Understanding in Geometry Questions
Automatically solving geometry questions is a longstanding AI problem. A geometry question typically includes a textual description accompanied by a diagram. The first step in solving geometry…