SQuAD: 100,000+ questions for machine comprehension of text¶
Authors: Pranav Rajpurkar; Jian Zhang; Konstantin Lopyrev; Percy Liang
Year: 2016
Venue: arXiv preprint
type: dataset collection: sok-references status: auto
Links: arXiv:1606.05250
Summary¶
Auto-extracted summary (unreviewed)
The paper introduces SQuAD, a reading-comprehension dataset of 107,785 crowdsourced question-answer pairs over 536 Wikipedia articles in which every answer is a text span of the passage, and analyzes the reasoning types required using dependency and constituency trees. A logistic-regression baseline reaches 51.0% F1 (40.4% exact match) on the test set, far below human performance of 86.8% F1 (77.0% exact match), establishing the dataset as a challenge benchmark.
Abstract¶
We present the Stanford Question Answering Dataset (SQuAD), a new reading comprehension dataset consisting of 100,000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to each question is a segment of text from the corresponding reading passage. We analyze the dataset to understand the types of reasoning required to answer the questions, leaning heavily on dependency and constituency trees. We build a strong logistic regression model, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%). However, human performance (86.8%) is much higher, indicating that the dataset presents a good challenge problem for future research. The dataset is freely available at https://stanford-qa.com
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | SQuAD dataset of 100,000+ span-based QA pairs on Wikipedia articles | llm | 0.90 |
| contribution | analysis of answer types and reasoning / syntactic divergence using dependency and constituency trees | llm | 0.90 |
| contribution | logistic regression baseline with feature ablations | llm | 0.90 |
| contribution | measurement of human performance on dev and test sets | llm | 0.90 |
| models | logistic regression | llm | 0.90 |
| models | sliding-window baseline | llm | 0.90 |
| datasets | SQuAD | llm | 0.90 |
| tasks | reading comprehension | llm | 0.90 |
| tasks | extractive question answering | llm | 0.90 |
| code url | https://worksheets.codalab.org/worksheets/0xd53d03a48ef64b329c16b9baf0f99b0c/ | llm | 0.90 |
| key result | dataset size: 107,785 QA pairs on 536 articles (SQuAD v1.0) | llm | 0.90 |
| key result | F1: 51.0% (logistic regression, test set) | llm | 0.90 |
| key result | exact match: 40.4% (logistic regression, test set) | llm | 0.90 |
| key result | F1: 86.8% (human performance, test set) | llm | 0.90 |
| key result | exact match: 77.0% (human performance, test set) | llm | 0.90 |
| key result | sentence selection accuracy: 79.3% (logistic regression selects the sentence containing the answer) | llm | 0.90 |
SoK reference [44]¶
P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ questions for machine comprehension of text,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, J. Su, K. Duh, and X. Carreras, Eds. Austin, Texas: Association for Computational Linguistics, Nov. 2016, pp. 2383–2392. [Online]. Available: https://aclanthology.org/ D16-1264/
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- model: Logistic regression 16 [p.1], CNN 3 [p.2], RNN / LSTM 1 [p.10]
- dataset: SQuAD 21 [p.1], ImageNet 2 [p.1], Penn Treebank 2 [p.1]
- task: Question answering 7 [p.2]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/sok-references/044_squad-100-000-questions-for-machine-comprehension-of-text.pdf |
sok-references | 10 | 446.1 KB | 5169f9424ad6 |
Database key: squad-100-000-questions-for-machine-comprehension-of-text · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set squad-100-000-questions-for-machine-comprehension-of-text COLUMN VALUE