Automatically constructing a corpus of sentential paraphrases¶
Authors: B. Dolan; C. Brockett
Year: 2005
type: dataset collection: sok-references status: auto
Links: link
Summary¶
Auto-extracted summary (unreviewed)
Describes the construction of the Microsoft Research Paraphrase Corpus (MRPC): 5,801 sentence pairs drawn from topic-clustered news data using heuristic extraction and an SVM classifier deliberately skewed toward over-identification, then hand-labeled by human judges as paraphrase or not. After adjudication 3,900 pairs (67%) were judged semantically equivalent; the paper also discusses the loose equivalence criterion and rater guidelines.
Abstract¶
An obstacle to research in automatic paraphrase identification and generation is the lack of large-scale, publiclyavailable labeled corpora of sentential paraphrases. This paper describes the creation of the recently-released Microsoft Research Paraphrase Corpus, which contains 5801 sentence pairs, each hand-labeled with a binary judgment as to whether the pair constitutes a paraphrase. The corpus was created using heuristic extraction techniques in conjunction with an SVM-based classifier to select likely sentence-level paraphrases from a large corpus of topicclustered news data. These pairs were then submitted to human judges, who confirmed that 67% were in fact semantically equivalent. In addition to describing the corpus itself, we explore a number of issues that arose in defining guidelines for the human raters.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | release of the MRPC corpus of 5,801 hand-labeled sentence pairs | llm | 0.95 |
| contribution | heuristic plus SVM-classifier pipeline for extracting candidate paraphrases from news clusters | llm | 0.95 |
| contribution | analysis of semantic-equivalence criteria and rater instructions | llm | 0.95 |
| models | SVM | llm | 0.95 |
| datasets | MRPC | llm | 0.95 |
| tasks | paraphrase identification | llm | 0.95 |
| limitations | loose definition of semantic equivalence: many pairs labeled equivalent diverge in content | llm | 0.95 |
| limitations | near-identical pairs (word Levenshtein distance below 8) were excluded | llm | 0.95 |
| key result | corpus size: 5,801 sentence pairs (MRPC) | llm | 0.95 |
| key result | positive rate: 3,900 pairs (67%) judged semantically equivalent (two judges, third resolves disagreements) | llm | 0.95 |
| key result | candidate pool: 20,574 pairs selected from 49,375 by classifier (before random sampling of 5,801 for annotation) | llm | 0.95 |
SoK reference [45]¶
B. Dolan and C. Brockett, “Automatically constructing a corpus of sentential paraphrases,” in Third International Workshop on Paraphrasing (IWP2005). Asia Federation of Natural Language Processing, January 2005. [Online]. Available: https://www.microsoft.com/en-us/research/publication/ automatically-constructing-a-corpus-of-sentential-paraphrases/
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: Leveled / somewhat HE 3 [p.4]
- model: SVM 5 [p.1]
- dataset: Penn Treebank 1 [p.6]
- task: Machine translation 5 [p.1], Genomics 2 [p.4]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/sok-references/045_automatically-constructing-a-corpus-of-sentential-paraphrase.pdf |
sok-references | 8 | 116.5 KB | 24be96310626 |
Database key: automatically-constructing-a-corpus-of-sentential-paraphrases · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set automatically-constructing-a-corpus-of-sentential-paraphrases COLUMN VALUE