TL;DR

Montero et al. present a TFHE-based framework that performs SGD training of logistic regression and small one-hidden-layer MLPs directly on encrypted data, using 4-6 bit integer quantization of weights, gradients, activations, and errors plus a recently introduced TFHE rounding operator to keep PBS bit-widths low [§1, §3]. On three tabular datasets (breast-cancer, mortality, heart) encrypted training reaches accuracies within ~1-3% of the fp32 plaintext baselines [§4.2, Table 2].

Problem and motivation

Training data is expensive and sensitive; collaborative training across multiple parties risks data leakage. Federated learning requires live client participation and is awkward for vertical splits and offline encrypted-at-rest data [§1]. The authors target a setting where data owners outsource training of LR or small NNs to a server that only sees ciphertext, supporting both horizontal and vertical data splits without an explicit alignment step [§1]. The threat model is implicit: the server is malicious/untrusted with respect to data confidentiality; mini-batch interaction with the data owner is allowed for fresh encryption (or replaced by bootstrapping for non-interactive training) [§3.5].

Key contributions

FHE setup

ML setup

Datasets

Dataset Task Size (train/test) Modality Notes
mortality [7] Binary classification 46582 examples, 20% held-out test Tabular 10 features, 2 classes (infant mortality, rural/urban US 2014)
breast-cancer [15] Binary classification 569 examples, 20% held-out test Tabular 30 features (Wisconsin diagnostic, OpenML ID 15)
heart [6] Binary classification Not stated, 20% held-out test Tabular 13 features (coronary artery disease)

Pipeline diagram

flowchart LR
    subgraph Calibration [Offline calibration]
        K[Plaintext synthetic data] --> Q[Quant params + LUTs]
        Q --> CG[ONNX integer graph]
        CG --> CC[TFHE compile + partitions]
    end
    subgraph Client
        X[Quantize X, w, b] --> E[Encrypt with sk]
    end
    subgraph Server
        S1[PBS matmul + add] --> S2[PBS sigmoid/quant LUT]
        S2 --> S3[Gradient + weight update]
    end
    subgraph ClientLoop [Client]
        D[Decrypt weights] --> R[Re-encrypt fresh]
    end
    CC -. eval keys .-> S1
    E -. ciphertext .-> S1
    S3 -. enc(w_new) .-> D
    R -. ciphertext .-> S1

Pipeline steps (text)

  1. Express the LR/MLP single-batch weight-update graph in PyTorch and export to ONNX [§3.1].
  2. Run offline quantization calibration on synthetic uniform data to fix per-tensor n_b-bit quantizers and fuse them into LUTs [§3.2].
  3. Compile the integer graph into TFHE partitions, each a multi-sum followed by a PBS; generate 128-bit-secure crypto parameters and bootstrap keys per partition [§3.3].
  4. Client quantizes data, weights, and biases and encrypts them; sends ciphertexts and evaluation keys to the server [§3.2].
  5. Server runs one SGD mini-batch (batch size 8) under TFHE — matmul via PBS-evaluated f_sq, sigmoid + quantization via fused LUT, error and gradient via PBS, weight update [§3.3, §4.1].
  6. The rounded-PBS operator strips low bits to keep PBS input bit-width small (Figure 3) [§3.4].
  7. Server returns encrypted updated weights; client decrypts to evaluate / early-stop and re-encrypts for the next mini-batch (or, alternatively, server bootstraps weights to stay non-interactive) [§3.5].
  8. Iterate until convergence (e.g., 20 batches on breast-cancer) [§4.2].

Architecture diagram

Logistic Regression (single-batch update graph)

flowchart LR
    X([X 1x64x8]) --> Q1[Quant Div B=1]
    W0([weight_0 1x64x8]) --> MM[MatMul]
    Q1 --> MM
    MM --> A1[Add bias_0]
    A1 --> SIG[Sigmoid via PBS]
    SIG --> SUB[Sub vs y]
    SUB --> DV[Div B=64]
    DV --> RS[ReduceSum axes=1]
    RS --> MUL1[Mul B=-1]
    MUL1 --> ADDB[Add bias_1]
    SUB --> TR[Transpose]
    TR --> MM2[MatMul]
    MM2 --> MUL2[Mul B=-1]
    MUL2 --> ADDW[Add weight_1]
    ADDB --> OB([new bias])
    ADDW --> OW([new weights])

MLP (one hidden layer)

flowchart LR
    I([Input d features]) --> FC1[FC hidden]
    FC1 --> H[Sigmoid via PBS LUT]
    H --> FC2[FC output]
    FC2 --> S[Sigmoid via PBS LUT]
    S --> L[Error / loss]
    L --> BW[Backprop via PBS LUTs]
    BW --> U([Updated w, b])

Note: the paper does not state the hidden width explicitly; #params is 930 for breast-cancer (30 features) and 165 for mortality (10 features), consistent with a hidden width of ~30 / ~14 respectively [Table 2].

Results

Headline (Table 2): on a held-out 20% test set, FHE-trained accuracies are within ~1-3% of fp32 baselines; latency reported for 16-thread execution on an 8-core processor [§4.2, Table 2].

Metric This paper Baseline Hardware
Breast-cancer LR (6b) accuracy 98.25% fp32: 99.12% 8-core, 16 threads [Table 2]
Breast-cancer LR (6b) batch / epoch latency 11.8 s / 0.23 h — 8-core, 16 threads [Table 2]
Breast-cancer MLP-930 (4b) accuracy 98.25% fp32: 99.12% 8-core, 16 threads [Table 2]
Breast-cancer MLP-930 (4b) batch / epoch latency 149 s / 2.94 h — 8-core, 16 threads [Table 2]
Mortality LR (6b) accuracy 90.09% fp32: 90.47% 8-core, 16 threads [Table 2]
Mortality LR (6b) batch / epoch latency 7.2 s / 11.6 h — 8-core, 16 threads [Table 2]
Mortality MLP-165 (4b) accuracy 87.25% fp32: 90.44% 8-core, 16 threads [Table 2]
Mortality MLP-165 (4b) batch / epoch latency 45 s / 72.78 h — 8-core, 16 threads [Table 2]
Heart LR (6b) accuracy 88.52% fp32: 89.47% 8-core, 16 threads [Table 2]
Heart LR (6b) batch / epoch latency 8.02 s / 0.08 h — 8-core, 16 threads [Table 2]
MLP throughput (WGC/s/T) 3 (breast-cancer) Glyph [12]: 24; Nandakumar [13]: 0.4 48-thread comparison for [12] [§4.2]
Batch-time grid (mortality, varying n_b, n_r) 1.49 s (n_b=2,n_r=3) up to 12.36 s (n_b=7,n_r=7) — 64-core machine [Figure 5]

Additional notes: PBS error rate is set to 1% per univariate evaluation; tightening to 2^-40 yields identical accuracy to 2 significant digits but doubles latency [§4.2]. Going from n_b = 5 to n_b = 6 roughly doubles latency [§4.2]. On breast-cancer, mini-batch convergence is reached in ~20 batches, ~3 minutes total — similar order to Kim et al. [11] which uses one big batch [§4.2].

Limitations and assumptions

Encrypted LR training: Bergamaschi et al. [1], Bonte & Vercauteren [3] (FV with fixed-Hessian), Han et al. [9] (CKKS, mini-batch packing + periodic bootstrapping), Kim et al. [11] (CKKS + Nesterov). Encrypted NN training: Glyph (Lou et al.) [12] (BGV + scheme-switching to TFHE for ReLU/SoftMax) and Nandakumar et al. [13] (BGV, two-layer MLP on MNIST/transfer-learning). Integer-arithmetic quantized training: WAGE [17], NITI [16], SWALP [18]. Quantization-aware integer inference: Jacob et al. [10]. TFHE compilation and parameter optimization: Bergerat et al. [2]; Stoian et al. [14] (which is the inference companion of this training framework).

Code and artifacts

Not released (no repository URL is mentioned in the paper).

Extra diagrams (optional)

Threat model

flowchart LR
    D[Data owner / client] -- enc(X,y,w0,b0) + eval keys --> S[Untrusted server]
    S -- enc(w_new, b_new) --> D
    D -- decrypt + early-stop + re-encrypt --> D

The client holds the secret key and quantization params; the server holds only ciphertexts and bootstrapping keys [§3.2, §3.3, §3.5].

Activation approximation

Sigmoid is not approximated by a polynomial; it is implemented as an integer LUT evaluated through PBS, fused with the surrounding quantization functions during calibration [§3, §3.2]. The TFHE rounding operator (Figure 3) is the key mechanism: to remove the LSB of a 5-bit accumulator it (i) shifts the LSB to the MSB, (ii) applies a 1-bit PBS to move it back, and (iii) subtracts the result from the original value [§3.4, Figure 3].

Open questions