Skip to content

Automatically constructing a corpus of sentential paraphrases

Authors: B. Dolan; C. Brockett
Year: 2005

type: dataset collection: sok-references status: auto

Links: link

Summary

Auto-extracted summary (unreviewed)

Describes the construction of the Microsoft Research Paraphrase Corpus (MRPC): 5,801 sentence pairs drawn from topic-clustered news data using heuristic extraction and an SVM classifier deliberately skewed toward over-identification, then hand-labeled by human judges as paraphrase or not. After adjudication 3,900 pairs (67%) were judged semantically equivalent; the paper also discusses the loose equivalence criterion and rater guidelines.

Abstract

An obstacle to research in automatic paraphrase identification and generation is the lack of large-scale, publiclyavailable labeled corpora of sentential paraphrases. This paper describes the creation of the recently-released Microsoft Research Paraphrase Corpus, which contains 5801 sentence pairs, each hand-labeled with a binary judgment as to whether the pair constitutes a paraphrase. The corpus was created using heuristic extraction techniques in conjunction with an SVM-based classifier to select likely sentence-level paraphrases from a large corpus of topicclustered news data. These pairs were then submitted to human judges, who confirmed that 67% were in fact semantically equivalent. In addition to describing the corpus itself, we explore a number of issues that arose in defining guidelines for the human raters.
Source: pdf

Extracted details

Field Value Source Confidence
contribution release of the MRPC corpus of 5,801 hand-labeled sentence pairs llm 0.95
contribution heuristic plus SVM-classifier pipeline for extracting candidate paraphrases from news clusters llm 0.95
contribution analysis of semantic-equivalence criteria and rater instructions llm 0.95
models SVM llm 0.95
datasets MRPC llm 0.95
tasks paraphrase identification llm 0.95
limitations loose definition of semantic equivalence: many pairs labeled equivalent diverge in content llm 0.95
limitations near-identical pairs (word Levenshtein distance below 8) were excluded llm 0.95
key result corpus size: 5,801 sentence pairs (MRPC) llm 0.95
key result positive rate: 3,900 pairs (67%) judged semantically equivalent (two judges, third resolves disagreements) llm 0.95
key result candidate pool: 20,574 pairs selected from 49,375 by classifier (before random sampling of 5,801 for annotation) llm 0.95

SoK reference [45]

B. Dolan and C. Brockett, “Automatically constructing a corpus of sentential paraphrases,” in Third International Workshop on Paraphrasing (IWP2005). Asia Federation of Natural Language Processing, January 2005. [Online]. Available: https://www.microsoft.com/en-us/research/publication/ automatically-constructing-a-corpus-of-sentential-paraphrases/

Listed on the reference page of the site.

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Leveled / somewhat HE 3 [p.4]
  • model: SVM 5 [p.1]
  • dataset: Penn Treebank 1 [p.6]
  • task: Machine translation 5 [p.1], Genomics 2 [p.4]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/sok-references/045_automatically-constructing-a-corpus-of-sentential-paraphrase.pdf sok-references 8 116.5 KB 24be96310626

Database key: automatically-constructing-a-corpus-of-sentential-paraphrases · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set automatically-constructing-a-corpus-of-sentential-paraphrases COLUMN VALUE