The dataset contains 9,356 science terms and, for each term, an average of 16,000 sentences that contain the term.

Science Terms and Sentences

The TQA dataset encourages work on the task of Multi-Modal Machine Comprehension (M3C) task. The M3C task builds on the popular Visual Question Answering (VQA) and Machine Comprehension (MC) paradigms by framing question answering as a machine comprehension task, where the context needed to answer questions is provided and composed of both text and images. The dataset constructed to showcase this task has been built from a middle school science curriculum that pairs a given question to a limited span of knowledge needed to answer it.


[![diagram](http://data.allenai.org/tqa/img/tqa_overview.png)](http://data.allenai.org/tqa/img/tqa_overview.png)

#### Construction

This dataset was constructed in large part from material freely available as part of [ck12.org's](http://www.ck12.org) open-source science curriculum. It is distributed under a Creative Commons Attribution – Non-Commerical 3.0 license.

#### Instructional Material

Lessons in the dataset each have limited amount of multimodal material that contains the information needed to answer that lesson's text and diagram questions. Textual material is broken down by subtopic within the lesson. Figures and instructional diagrams are linked from the topic in which they appear.

#### Questions

The dataset is split into a training, validation and test set at the lesson level. The training set consists of 666 lessons and 15,154 questions, the validation set consists of 200 lessons and 5,309 questions and the test set consists of 210 lessons and 5,797 questions. On occasions, multiple lessons have an overlap in the concepts they teach. Care has been taken to group these lessons before splitting the data, so as to minimize the concept overlap between splits.

Questions are separated by type in the dataset (nested under diagramQuestions and nonDiagramQuestions), and are also distinguished by global id prefixes indicating an associated diagram (DQ\_ _and NDQ\__).

#### Images

Images in the dataset are divided into four types:

question\_images (diagrams referred to in diagram questions)

abc\_question\_images (question diagrams generated with letter labels)

teaching\_images (diagrams which have detailed descriptions)

textbook\_images (images from the instructional material)

Images are referenced in the dataset as relative paths to one of these directories.

#### Structure

Here is the structure of the dataset json files for the train, test, and validation sets:

    [  ### at the top level is a list of "lessons" that are the atomic unit of the dataset
        { ### within a lesson are several fields, including questions and their related instructional material
                "globalID": "global id string",
                "lessonName": "name of lesson",
                "questions": {
                "nonDiagramQuestions": {
                    "question global ID": {
                        "answerChoices": {
                            "a": {
                                "idStructural": "a.",
                                "processedText": "cleaned answer option string",
                                "rawText": "unprocessed answer option given in question"
                            },
                            .
                            .
                            ... additional answer options                            
                        },
                        "beingAsked": {
                            "processedText": "cleaned question string",
                            "rawText": "unprocessed question being asked"
                        },
                        "correctAnswer": {
                            "processedText": "cleaned correct answer string",
                            "rawText": "correct answer to question"
                        },
                        "globalID": "global id number",
                        "idStructural": "number identifier from source material",
                        "questionType": "question type"
                        }
                        .
                        .
                        .... additional questions
                }
                "diagramQuestions": {
                            The same fields as non-diagram questions, plus:
                            "imagePath": "relative local path to image file"
                             "imageName": "image name"
                },
                "instructionalDiagrams": {
                        "global_id": {
                             "global id": "global id number",
                             "imagePath": relative local path to image file",
                            "imageName": "image name",
                            "rawText": "description of diagram",
                            "processedText": "processed description of diagram"
                        },
                        ...
                },
                "diagramAnnotations": {
                    "diagram_name": [
                    {
                        "text": "the ground truth content of the text box",
                        "rectangle": "the location of the text "
                    },
                    ...
                },
                "topics": {
                        "Global Topic ID": {
                            "global ID": global id string,
                         "content": {
                                "figures": [
                                {
                                       "caption": "figure caption text",
                                        "imagePath}": "relative local path to image file"
                                 }
                                ],
                            "text": "Paragraph sized text explaining topic"
                                    },
                                    "orderID": "order of appearance within lesson",
                                    "mediaLinks": {[
                                list of youtube/ other video links extracted from the text]
                                            }
                                      }
                              }
                "adjunctTopics": {
                        "Vocabulary": {
                            dictionary of lesson vocabulary words and their definitions.
                        },
                        "section name": {
                             "content": {
                                "figures": [
                                {
                                       "caption": "figure caption text",
                                        "imagePath": "relative local path to image file"
                                 }
                                ],
                            "text": "Paragraph sized text explaining topic"
                                    },
                                    "orderID": "order of appearance within lesson",
                                    "mediaLinks": {[
                                list of youtube/ other video links extracted from the text]
                                            }
                                      }
                              }
                        }
    
                }
              }                
        .
        .
        ... repeated for additional lessons
       ]
    

*   Top-Level Lesson: This is the atomic unit of the dataset, and the level at which questions are paired to instructional material, e.g "History of Life on Earth". The important components of a lesson are described below.
    
    *   questions: questions that should be answerable given the contents of the lesson.
        
        *   nonDiagramQuestions: questions with no accompanying diagram needed to answer
        *   diagramQuestions: questions with an accompanying diagram needed to answer
            *   questionType: format of the question, multiple choice, fill-in-the-blank, true/false, diagram multiple choice
            *   beingAsked: text of the question being posed
            *   idStructural: the original number or letter identifier in the source, e.g. "1.", "2.", "A."
            *   globalID: dataset-unique id number assigned to the question
            *   answerChoices: possible answer options- the number and form will depend on the question type
            *   correctAnswer: the correct answer given for the question
            *   imagePath: relative path to image file from the dataset root directory.
    *   topics:
        
        *   topic name: name string, e.g. Carboniferous Period, Cenozoic Era, Cretaceous Period
        *   content: instructional content extracted from source material
            *   globalID: dataset-unique id number assigned to the topic
            *   text: paragraph sized text blurb introducing and explaining topic
            *   figures: diagram images associated with topic, if any
            *   caption: image caption
            *   imagePath: relative path to image file from the dataset root directory.
            *   orderID: order of appearance within lesson
    *   adjunctTopics:
        
        *   section type: vocabulary, summary, introduction, apply concepts, etc.
        *   content: instructional content extracted from source material
        *   text: paragraph sized text blurb introducing and explaining topic
        *   figures: diagram images associated with topic, if any
            *   caption: image caption
            *   imagePath: relative path to image file from the dataset root directory.
            *   orderID: order of appearance within lesson
    *   diagramAnnotations
        
        *   diagram name: name of diagram image file (matching imageName field in diagramQuestions and instructionalDiagrams)
            *   A list of groundtruth diagram text boxes with
                *   text: human transcribed text
                *   rectangle: location of text bounding box on the image in the form: \[lower left coordinate, upper right coordinate\]
    *   instructionalDiagrams
        
        *   diagram name:
            *   "imagePath": relative path to image file from the dataset root directory.
            *   "imageName": name of the diagram image file,
            *   "rawText": human written description of diagram,
            *   "processedText": normalized text of the description
    *   metaLessonID: an identifier that links lessons that are similar in their concepts and subject matter. This is to prevent similar material being placed in both test and train.

The TextbookQuestionAnswering (TQA) dataset is drawn from middle school science curricula. It consists of 1,076 lessons from Life Science, Earth Science and Physical Science textbooks. This includes 26,260 questions, including 12,567 that have an accompanying diagram.

Textbook Question Answering (TQA)

1,076 textbook lessons, 26,260 questions, 6229 images

This dataset is designed to demonstrate Explicit Semantic Ranking (ESR), a new ranking technique that leverages knowledge graph embedding. Analysis of the query log from our academic search engine, SemanticScholar.org, reveals that a major error source is its inability to understand the meaning of research concepts in queries. To addresses this challenge, ESR represents queries and documents in the entity space and ranks them based on their semantic connections from their knowledge graph embedding. Experiments demonstrate ESR's ability in improving Semantic Scholar's online production system, especially on hard queries where word-based ranking fails.

#### Citation

If you find this dataset helpful in your work, please cite:

    @inproceedings{xiong2017ESR,
    title={Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding},
    author={Xiong, Chenyan and Power, Russell and Callan, Jamie},
    booktitle={Proceedings of the 26th International Conference on World Wide Web (WWW 2017)},
    note={To appear},
    year={2017},
    organization={ACM}
    };

#### Structure

In the zip file you will find the following files and folders:

s2\_query.json contains the queries used in this paper. Each line of it is a json format dictionary, with the following format:

    {
      "qid": "the query id",
      "query": "the query string",
      "ana": {
        the annotated entity id and frequency
      }
    }
    
The entities are from Freebase. Please refer to the final dump of Freebase to get more information about these entities.
    

s2.trec is the TREC format ranking files. It contains the ranking lists from semanticscholar.org's production search engine (as of 2016 summer).

s2\_doc.json contains the candidate documents. Each line of it is a json format dictionary. Its fields include:

    docno: the doc id
    title
    keyPhrase: the automatically extracted key phrases for this paper.
    paperAbstract: paper abstract
    venue
    numCitedBy: number of citations
    numKeyCitations: number of key citations. Key citation means the other paper considers this one as a very important related work. It is from semanticsholar's production system.
    Ana: the annotation of each of the title, paperAbstract, and body field.
    
Due to copyright restrictions, we are not allowed to release the body text. Please check https://api.semanticscholar.org/ to get the full corpus and more information about each document.

s2.qrel is the relevance judgments for these queries. It was labeled by the first two authors. Judging the relevance of computer science papers is very hard. We have to read many papers' abstract or even introductions ourselves before making any reasonable judgments. The current size of labels is limited. Keep updated with SemanticScholar.org for future possible benchmark release.

ranking\_res folder includes the ranking results of all baselines, develop methods, and alternative methods in the experiments and analysis of this paper. Feel free to conduct future experiments based on them.

knowledge\_graph\_embedding folder contains the entity embeddings trained using our knowledge graph. It is in Google word2vec format.

#### Queries

    queryID  query
    -------  ------------------------------------------------------------
    1        deep learning
    2        artificial intelligence
    3        information retrieval
    4        machine learning
    5        question answering
    6        noun phrases
    7        penn treebank
    8        speech recognition
    9        data mining
    10       computer vision
    11       reinforcement learning
    12       natural language
    13       autoencoder
    14       ontology
    15       sentiment analysis
    16       sap
    17       lstm
    18       natural language processing
    19       semantic web
    20       mooc
    21       human computer interaction
    22       eye movement clustering
    23       semantic relations
    24       efficient estimation of word representations in vector space
    25       big data
    26       audio visual fusion
    27       object detection
    28       gfdm
    29       neural network
    30       generalized extreme value
    31       information geometry
    32       image panorama video
    33       data science
    34       semantic parsing
    35       augmented reality
    36       imbalanced data
    37       recommender system
    38       inverse reinforcement learning mixture
    39       transfer learning
    40       cnn
    41       dynamic programming segmentation
    42       natural language interface
    43       genetic algorithm
    44       prolog
    45       contact prediction
    46       wifi malware
    47       nsdi machine learning
    48       forensics and machine learning
    49       words to speech
    50       information theory
    51       morphology morphological
    52       category theory
    53       graph theory
    54       smart thermostat
    55       exploit vulnerability
    56       reinforcement learning and video game
    57       system health management
    58       spatial multi agent systems
    59       service composition
    60       mobile payment
    61       3 axis gantry
    62       softmax categorization
    63       cost aggregation
    64       chinese dialect
    65       depth camera
    66       mobile tcp traffic analysis
    67       collective learning
    68       robust production planning
    69       memory hierarchy
    70       hashing
    71       comparable corpora
    72       knowledge graph
    73       social media
    74       deep learning surveillance
    75       cryptography
    76       parametric max flow
    77       deep reinforcement learning
    78       varying weight grasp
    79       dirichlet process
    80       word embedding
    81       graph drawing
    82       robust principal component analysis
    83       differential evolution
    84       seq2seq
    85       document logical structure
    86       duality
    87       variable neighborhood search
    88       urban public transportation systems
    89       edx coursera
    90       fdir
    91       cryptography key management
    92       ontology construction
    93       go game
    94       personality trait
    95       sparse learning
    96       directed hypergraph
    97       inventory management
    98       clojure
    99       ontology semantic web
    100      convolutional neural network time series

This is the dataset for the paper Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding. It includes the query log used in the paper, relevance judgements for the queries, ranking lists from Semantic Scholar, candidate documents, entity embeddings trained using the knowledge graph, and baselines, development methods, and alternative methods from the experiments.

Explicit Semantic Ranking Dataset

The Aristo Tuple KB contains a collection of high-precision, domain-targeted (subject,relation,object) tuples extracted from text using a high-precision extraction pipeline, and guided by domain vocabulary constraints.

In addition to the dataset, you can download the [Aristo MINI Corpus](http://ai2-website.s3.amazonaws.com/data/Aristo-Mini-Corpus-Dec2016.zip) that is used for measuring "comprehensiveness".

#### Citation

If you use this data in your research please refer to the tuple KB by its release name and date ("Aristo Tuple KB v5 - Mar 2017 Release"), and provide an acknowledgement to Allen Institute for AI.

#### Release Notes

Previous releases of the Aristo Tuple KB may be found below:

*   [Aristo Tuple KB v4 (March 2017)](https://s3-us-west-2.amazonaws.com/ai2-website/data/artisto-tuple-kb/aristo-tuple-kb-v4-mar2017.zip) (NB: this is the first of two March 2017 releases)
*   [Aristo Tuple KB v3 (January 2017)](https://s3-us-west-2.amazonaws.com/ai2-website/data/artisto-tuple-kb/aristo-tuple-kb-v3-jan2017.zip)
*   [Aristo Tuple KB v2 (December 2016)](https://s3-us-west-2.amazonaws.com/ai2-website/data/artisto-tuple-kb/aristo-tuple-kb-v2-dec2016.zip)
*   [Aristo Tuple KB v1 (November 2016)](https://s3-us-west-2.amazonaws.com/ai2-website/data/artisto-tuple-kb/aristo-tuple-kb-v1-nov2016.zip) (NB: this is the first of two November 2016 releases)
*   [Aristo Tuple KB v0 (November 2016)](https://s3-us-west-2.amazonaws.com/ai2-website/data/artisto-tuple-kb/aristo-tuple-kb-v0-nov2016.zip)

#### Contact

If you have any other questions or feedback for us about this data, please contact [bhavanad@allenai.org](mailto:bhavanad@allenai.org).

The Aristo Tuple KB contains 294,000 high-precision, domain-targeted (subject,relation,object) tuples extracted from text using a high-precision extraction pipeline, and guided by domain vocabulary constraints.

Aristo Tuple KB

The Aristo Mini corpus contains 1,197,377 (very loosely) science-relevant sentences drawn from public data. It provides simple science-relevant text that may be useful to help answer elementary science questions.

Aristo Mini Corpus

#### Paper Appendix

The paper appendix describes the semantic parser and execution model's features:

[Appendix: Semantic Parsing to Probabilistic Programs forSituated Question Answering](http://ai2-website.s3.amazonaws.com/publications/emnlp2016-appendix.pdf)

#### Code

An implementation of **Parsing to Probabilistic Programs** (P3) is available as part of [jklol](https://github.com/jayantk/jklol).

See the [instructions for running the SCENE experiment](https://github.com/jayantk/jklol/tree/master/src/com/jayantkrish/jklol/experiments/p3).

The foodwebs dataset contains 5000 questions about 500 food web diagrams. Each diagram has annotations from a computer vision system and each question is annotated with a logical form.

Foodwebs

5000 questions about 500 food web diagrams.

This dataset contains gold explanation sentences supporting 363 science questions, relation annotation for a subset of those explanations, and a graphical annotation tool with annotation guidelines.

Explanations for Science Questions

AI2D is a dataset of illustrative diagrams for research on diagram understanding and associated question answering.

AI2 Diagram Dataset

These questions were created using the "AI2 Elementary School Science Questions (No Diagrams)" data set by changing all of the incorrect answer options of each question with some other related word. This dataset can be a good measure of robustness for QA systems when being testing on modified questions.

Perturbed Version of AI2 Elementary School Science Questions (No Diagrams)

#### Code

PAL is available as part of [jklol](https://github.com/jayantk/jklol).

[Instructions for running lexicon learning](https://github.com/jayantk/jklol/tree/master/src/com/jayantkrish/jklol/ccg/lexinduct).

774 food chain questions designed to imitate actual questions from the New York State Grade 4 Regents Exam.

Research