Metrics¶
Registry¶
AVAILABLE_METRICS maps each metric key (as used by ecgen-eval eval -m KEY) to its class.
Base classes¶
MetricResult
dataclass
¶
MetricResult(metric_name: str, score: float, higher_is_better: bool = False, per_lead_scores: Dict[str, float] = dict(), extra: Dict[str, Any] = dict(), description: str = '', reference: str = '')
Standardised result returned by every metric.
Attributes:
| Name | Type | Description |
|---|---|---|
metric_name |
str
|
Short identifier (e.g. "mmd", "dtw"). |
score |
float
|
Primary scalar score. Convention: lower = more similar to real data,
unless |
higher_is_better |
bool
|
If True, a higher score means better quality. |
per_lead_scores |
dict
|
Optional per-lead breakdown {lead_name: score}. |
extra |
dict
|
Any additional data the metric wants to expose (e.g. p-values, per-sample distances) for rich report rendering. |
description |
str
|
One-line human-readable description of what was measured. |
reference |
str
|
Citation for the metric. |
BaseMetric
¶
Bases: ABC
Abstract base class all metrics must inherit from.
Subclasses implement :meth:compute which receives two arrays:
- real : np.ndarray of shape (N_r, L, T)
- synth : np.ndarray of shape (N_s, L, T)
and returns a :class:MetricResult.
compute
abstractmethod
¶
compute(real: ndarray, synth: ndarray, lead_names: Optional[List[str]] = None) -> MetricResult
Compute the metric.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
real
|
np.ndarray shape (N_r, L, T)
|
|
required |
synth
|
np.ndarray shape (N_s, L, T)
|
|
required |
lead_names
|
list of str
|
|
None
|
Returns:
| Type | Description |
|---|---|
MetricResult
|
|
Source code in ecgen_eval/metrics/base.py
Fréchet utilities¶
Shared Fréchet Distance math used by FDMetric and FEDMetric.
FD = ‖μ_X − μ_Y‖² + Tr(Σ_X + Σ_Y − 2·(Σ_X Σ_Y)^½)
frechet_distance
¶
Compute the Fréchet Distance between two sets of feature vectors.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
(ndarray, shape(N, D))
|
Feature matrix for the first distribution (e.g. real). |
required |
Y
|
(ndarray, shape(M, D))
|
Feature matrix for the second distribution (e.g. synthetic). |
required |
eps
|
float
|
Small value added to the covariance diagonal for numerical stability. |
1e-06
|
Returns:
| Type | Description |
|---|---|
float
|
Fréchet Distance (lower is better; 0 means identical distributions). |
Source code in ecgen_eval/metrics/_frechet_utils.py
_matrix_sqrt
¶
Symmetric matrix square root via eigendecomposition.
Source code in ecgen_eval/metrics/_frechet_utils.py
Metric classes¶
MMDMetric
¶
Bases: BaseMetric
MMD² computed per lead on a handcrafted statistical feature vector, then averaged across leads to yield a single scalar score.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kernel_bandwidths
|
list of float
|
Multiplicative factors applied to the median-heuristic bandwidth. Default: [0.1, 1.0, 10.0] (multi-scale RBF). |
None
|
n_subsample
|
int
|
Maximum number of samples drawn from each dataset per lead before computing the kernel matrix (for speed). Default: 1000. |
1000
|
Source code in ecgen_eval/metrics/mmd.py
compute
¶
compute(real: ndarray, synth: ndarray, lead_names: Optional[List[str]] = None) -> MetricResult
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
real
|
ndarray
|
|
required |
synth
|
(N_s, L, T)
|
|
required |
Source code in ecgen_eval/metrics/mmd.py
FDMetric
¶
Bases: BaseMetric
Fréchet Distance between Gaussian fits to ECG feature vectors per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_subsample
|
int
|
Maximum samples per set. Default: 2000. |
2000
|
eps
|
float
|
Small regularisation term added to covariance diagonal for stability. |
1e-06
|
Source code in ecgen_eval/metrics/fd.py
FEDMetric
¶
FEDMetric(backbone: str = 'ecgfounder', device: str = 'cpu', batch_size: int = 32, n_subsample: Optional[int] = None, eps: float = 1e-06)
Bases: BaseMetric
Fréchet ECG Distance using deep foundation-model embeddings.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
backbone
|
str
|
Which foundation model to use as the feature extractor.
One of |
'ecgfounder'
|
device
|
str
|
PyTorch device string, e.g. |
'cpu'
|
batch_size
|
int
|
Number of recordings per encoder forward pass. |
32
|
n_subsample
|
int or None
|
If set, randomly subsample at most this many recordings from each set
before computing embeddings (useful for large datasets). Default: |
None
|
eps
|
float
|
Regularisation term added to covariance diagonal for numerical stability. |
1e-06
|
Notes
Model weights are downloaded from HuggingFace on the first call to
compute(). Set the HF_HOME environment variable to control
the cache directory.
Source code in ecgen_eval/metrics/fed.py
compute
¶
compute(real: ndarray, synth: ndarray, lead_names: Optional[List[str]] = None) -> MetricResult
Compute the Fréchet ECG Distance between real and synth.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
real
|
(ndarray, shape(N, L, T))
|
|
required |
synth
|
(ndarray, shape(M, L, T))
|
|
required |
lead_names
|
list[str] or None
|
Ignored for FED (embeddings are global, not per-lead). |
None
|
Returns:
| Type | Description |
|---|---|
MetricResult
|
|
Source code in ecgen_eval/metrics/fed.py
SWDMetric
¶
Bases: BaseMetric
SWD on a 7-dimensional statistical feature space, averaged per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_projections
|
int
|
Number of random 1-D projections. Default: 200. |
200
|
n_subsample
|
int
|
Max samples per lead. Default: 1000. |
1000
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/swd.py
PRDistMetric
¶
Bases: BaseMetric
Precision/Recall for distributions, per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
k
|
int
|
k for k-NN manifold estimation. Default: 3. |
3
|
n_subsample
|
int
|
Max samples per lead. Default: 1000. |
1000
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/pr_dist.py
DTWMetric
¶
Bases: BaseMetric
Mean pairwise DTW distance between random (real, synthetic) pairs per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_pairs
|
int
|
Number of (real, synthetic) pairs to sample. Default: 200. |
200
|
downsample
|
int
|
Take every |
5
|
window
|
float
|
Sakoe-Chiba band as a fraction of signal length (0–1). Default: 0.1. |
0.1
|
seed
|
int
|
Random seed for reproducibility. |
42
|
Source code in ecgen_eval/metrics/dtw.py
compute
¶
compute(real: ndarray, synth: ndarray, lead_names: Optional[List[str]] = None) -> MetricResult
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
real
|
ndarray
|
|
required |
synth
|
(N_s, L, T)
|
|
required |
Source code in ecgen_eval/metrics/dtw.py
PRDMetric
¶
Bases: BaseMetric
PRD computed per lead between each real recording and its nearest synthetic neighbour in the flattened feature space, averaged over real samples.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_subsample
|
int
|
Cap on number of real/synthetic samples used (for speed). Default: 500. |
500
|
seed
|
int
|
Random seed when subsampling. |
42
|
Source code in ecgen_eval/metrics/prd.py
compute
¶
compute(real: ndarray, synth: ndarray, lead_names: Optional[List[str]] = None) -> MetricResult
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
real
|
ndarray
|
|
required |
synth
|
(N_s, L, T)
|
|
required |
Source code in ecgen_eval/metrics/prd.py
MAEMetric
¶
Bases: BaseMetric
Nearest-neighbour MAE and RMSE per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_subsample
|
int
|
Max samples per lead for speed. Default: 500. |
500
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/mae.py
PSNRMetric
¶
Bases: BaseMetric
Nearest-neighbour PSNR averaged per lead and across leads.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_subsample
|
int
|
Maximum samples drawn from each dataset per lead. Default: 500. |
500
|
seed
|
int
|
Random seed for subsampling. Default: 42. |
42
|
Source code in ecgen_eval/metrics/psnr.py
SSIMMetric
¶
Bases: BaseMetric
Global 1-D SSIM averaged per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
k1
|
float
|
Stability constants. Default: 0.01, 0.03 (standard values). |
0.01
|
k2
|
float
|
Stability constants. Default: 0.01, 0.03 (standard values). |
0.01
|
n_subsample
|
int
|
Max samples per lead for speed. Default: 500. |
500
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/ssim.py
SQIMetric
¶
Bases: BaseMetric
Template-correlation SQI per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_subsample
|
int
|
Max samples per lead. Default: 1000. |
1000
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/sqi.py
PSDMetric
¶
Bases: BaseMetric
Jensen-Shannon divergence between mean normalised PSDs per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fs
|
float
|
Sampling frequency in Hz. Default: 500. |
500.0
|
nperseg
|
int
|
Welch segment length. Default: min(256, T). |
None
|
n_subsample
|
int
|
Cap on samples used per set. Default: 1000. |
1000
|
Source code in ecgen_eval/metrics/psd.py
compute
¶
compute(real: ndarray, synth: ndarray, lead_names: Optional[List[str]] = None) -> MetricResult
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
real
|
ndarray
|
|
required |
synth
|
(N_s, L, T)
|
|
required |
Source code in ecgen_eval/metrics/psd.py
SpectralEntropyMetric
¶
SpectralEntropyMetric(fs: float = 500.0, nperseg: Optional[int] = None, n_subsample: int = 1000, seed: int = 42)
Bases: BaseMetric
Spectral entropy distribution comparison per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fs
|
float
|
Sampling frequency in Hz. Default: 500. |
500.0
|
nperseg
|
int or None
|
Welch segment length. None → min(256, T). Default: None. |
None
|
n_subsample
|
int
|
Max samples per lead. Default: 1000. |
1000
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/spectral_entropy.py
HRVMetric
¶
Bases: BaseMetric
HRV statistics comparison per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fs
|
float
|
Sampling frequency in Hz. Default: 500. |
500.0
|
n_subsample
|
int
|
Max samples per lead. Default: 1000. |
1000
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/hrv.py
HeartRateMetric
¶
Bases: BaseMetric
Heart-rate distribution comparison per lead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fs
|
float
|
Sampling frequency in Hz. Default: 500. |
500.0
|
n_subsample
|
int
|
Max samples per lead. Default: 1000. |
1000
|
seed
|
int
|
Random seed. Default: 42. |
42
|
Source code in ecgen_eval/metrics/heart_rate.py
NNDistMetric
¶
Bases: BaseMetric
NND and Authenticity computed per lead on a 7-feature statistical space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_subsample
|
int
|
Max samples per lead. Default: 1000. |
1000
|
seed
|
int
|
Random seed. Default: 42. |
42
|