PhysioNet/CinC Challenge 2017 (AF Classification)

8,528 single-lead recordings from a consumer handheld device, each labelled normal / AF / other rhythm / too noisy to classify. This is the reference benchmark for atrial fibrillation detection from consumer ECG rather than clinical 12-lead, and the only dataset in this catalogue whose fourth class is “unusable signal” — signal quality is part of the task, not a preprocessing step to get past.

Every recording was taken by a member of the public who had bought one of three generations of AliveCor’s single-channel device and held one electrode in each hand, giving a nominal lead I (LA-RA) equivalent. The device recorded for an average of 30 s, transmitted the trace acoustically to a phone over a 19 kHz carrier, and stored it at 300 Hz / 16 bits with a 0.5–40 Hz passband.

The channel is called ECG, not I, and that is deliberate. The paper states that “many of the ECGs were inverted (RA-LA) since the device did not require the user to rotate it in any particular orientation”, and no record says which. ECGBench declares the source’s own channel name so nobody stacks this with 12-lead lead I by name while an unknown fraction of the traces carry the opposite sign.

Record length varies and is not a nuisance variable. 2,714 to 18,286 samples — 9.05 s to 60.95 s, in 1,487 distinct lengths — so records cannot be batched without window=, and the window has to fit the shortest. Length also correlates with the label: noisy recordings average 24.4 s against 32–34 s for the other three classes. A model fed whole records can learn duration instead of rhythm.

This is the public training set only. The challenge used 12,186 recordings; the other 3,658 are the hidden test set, which was never released. Nothing here reproduces the challenge’s own scoring split — see “The shipped validation/ directory is not a split” below, which is the trap that follows from that.

Every figure on this page was recomputed from the shipped files, after verifying all 17,066 files under training/ against the release’s own MD5SUMS and all 610 under validation/ against its SHA256SUMS — every one matches.

The catalogue previously said 12,186 records. That is the figure the paper’s abstract gives, and it counts the hidden test set: 8,528 public + 3,658 private. Only the 8,528 were ever released, so that is what ECGBench partitions and what this page now states.

patients is — because no patient identifier exists, and unlike most one-record-per-patient datasets this one cannot even assert that. The recordings came from members of the public who had bought a handheld device, and nothing in the release says whether anyone contributed more than one. See “Folds are stratified but ungrouped” below.

The class table is single-label — every record carries exactly one of the four codes, none carries two, and none carries none — so the column sums to the record total, and the stratification label is the training target. That is unusual here; most ECGBench datasets need a reduction for folds that you must not train on.

The challenge relabelled its data twice mid-competition, using the entrants’ own disagreement to find the records it trusted least: the whole dataset was ranked by how much the top-performing algorithms disagreed, and eight ECG experts independently relabelled the 1,129 most contentious recordings. Over those, Fleiss’ κ was 0.245 — “fair agreement” — so label noise here is substantial and acknowledged by the organisers rather than hypothetical.

Four REFERENCE-v<N>.csv files ship, and their numbers do not match the paper’s V1/V2/V3. Matching the shipped training counts against the paper’s Table 2 gives:

So “v1” names a different file depending on whether you read the paper or the directory, and one shipped version has no name in the paper at all. REFERENCE.csv is byte-identical to REFERENCE-v3.csv, so v3 is the label. ECGBench names its columns after the shipped files — class_code_v0 … class_code_v3 — because those are what a reader can verify on disk.

412 of the 8,528 labels changed between v0 and v3, and the movement is almost entirely into the noisy class:

233 records moved into noisy and none moved out — the second pass was a visual recheck for recordings too noisy to read, and it only ever added. n_distinct_labels counts how many labels a record was ever given: 8,104 records were never relabelled, 418 carry two labels across the four versions and 6 carry three. Filtering on it is the closest this release comes to an annotation-confidence measure.

The release contains a validation/ directory of 300 records, and it is not held-out data. All 300 of its .mat files are byte-identical to their training/ counterparts — checked, all 300 — and the paper says what it is for: “Validation was 300 records (3.5%) of training set just to ensure the algorithm produced the expected results.”

Using it as a test set means evaluating on training data. ECGBench therefore sets has_predefined_splits: false, lets those records take part in the ten folds like any other, and flags them as in_challenge_validation_subset so they can be excluded from a comparison against published challenge numbers:

ds.labels_df[~ds.labels_df.in_challenge_validation_subset]

The real 3,658-record test set was never published, so the challenge’s own scoring split cannot be reproduced from this release at all. Numbers computed on ECGBench folds are not comparable to the challenge leaderboard.

Every one of the 8,528 headers is exactly two lines with no # comment of any kind — no age, no sex, no subject id, no clinical text, nothing. So patient_id_column is null and the engine uses plain StratifiedKFold.

This is worse than the usual “one record per patient, so grouping is moot”. The contributors were members of the public using their own devices, and nothing in the release says whether one person recorded once or fifty times. A repeat contributor would straddle folds undetectably, and no information in the files could detect it. Treat the resulting generalisation estimate as an upper bound.

The header timestamp is de-identified rather than removed, and is not a wall clock: the date is 1/<month>/2000 in every record — day always 1, year always 2000, only the month varying — and the hour field runs 0–12 for 8,378 records and 21–23 for the other 150, which is not a 24-hour distribution. header_timestamp is exposed verbatim so nobody has to guess; do not read it as a recording time.

A04894 is the only exclusion, for amplitude_outlier: its trace runs to −10.636 mV, just past the ±10 mV range. The device’s stated dynamic range is ±5 mV and the extremes over all 8,528 records are −10.636 mV to +8.318 mV, so this is a genuine rail-hitting excursion on a handheld recording, not a units error. The range was left at the standard ±10 mV rather than widened to make that record disappear — which is what amplitude_range_mv is for. Nothing exceeds ±20 mV, so a mis-scaled copy (microvolts, or the gain dropped) would still be caught on its first record.

Two other checks are worth explaining because they cannot fire here:

• truncated_signal is disabled, by leaving expected_samples empty. There are 1,487 distinct record lengths between 2,714 and 18,286 samples, so no single threshold separates a truncated record from a short one, and any threshold would drop thousands of sound records.
• nan_values and flat_line never fire. No record contains a NaN sample, and the lowest per-record standard deviation anywhere in the release is 0.032 mV — noisy, but never flat.

All 8,528 records read, so corrupt_header is clean too. Fold membership is identical between original/ and clean/; clean/ is the same partition minus one row.

No related: edge is declared, and the reason is structural rather than merely unverified. This is the only AliveCor single-lead cohort in the catalogue: the PhysioNet/CinC Challenge 2020 and 2021 releases that bundle other datasets are 12-lead meta-datasets built from eight clinical cohorts (CPSC, PTB, PTB-XL, Georgia, Chapman-Shaoxing, Ningbo, St Petersburg INCART), none of which is this one, and no other release in the catalogue redistributes these recordings.

The one genuine relative is the 3,658-record hidden test set, which is not public and so cannot be an entry here.