Machine Learning Training on Encrypted Data with TFHE¶
Authors: Luis Montero; Jordan Frery; Celia Kherfallah; Roman Bredehoft; Andrei Stoian
Year: 2024
Venue: Proceedings of the 10th ACM International Workshop on Security and Privacy Analytics
type: technique collection: ppml status: auto
Links: DOI
Summary¶
Auto-extracted summary (unreviewed)
This short paper from Zama presents a TFHE-based framework for outsourcing the training of quantized machine-learning models on encrypted data, using programmable bootstrapping (including a rounded PBS trick) and compiled crypto-system parameters, aimed at collaborative settings with horizontally or vertically split data. Training logistic regression and one-hidden-layer MLPs on breast-cancer, mortality and heart datasets reaches accuracy within about 2% of fp32 plaintext training (e.g., 98.25% vs 99.12% for logistic regression on breast-cancer), with batch latencies of 7-149 s and epoch times from 0.08 h to 72.78 h, and a throughput of 3 weight-gradient computations per second per thread versus 0.4 for one prior work.
Abstract¶
We present an approach for outsourcing the training of machine learning (ML) models while preserving data confidentiality from malicious parties. We use fully homomorphic encryption (FHE) to build a unified training framework that works on encrypted data and learns quantized ML models. Our approach finds future applications in collaborative settings involving multiple parties working on confidential data, which can be horizontally or vertically split between data owners. We train logistic regression and multilayer perceptrons on several datasets and show results that are comparable to the state-of-the-art.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Unified TFHE training framework for quantized logistic regression and MLPs on encrypted data | llm | 0.85 |
| contribution | Rounded programmable bootstrapping to drop least-significant bits cheaply | llm | 0.85 |
| contribution | Compilation-based selection of crypto-system parameters at 128-bit security | llm | 0.85 |
| contribution | Study of weight bit-width and quantization scaling trade-offs between accuracy and latency | llm | 0.85 |
| schemes | TFHE | llm | 0.85 |
| models | logistic regression | llm | 0.85 |
| models | MLP (1 hidden layer) | llm | 0.85 |
| datasets | breast cancer (OpenML) | llm | 0.85 |
| datasets | mortality | llm | 0.85 |
| datasets | heart | llm | 0.85 |
| tasks | encrypted ML training | llm | 0.85 |
| tasks | binary classification | llm | 0.85 |
| tasks | collaborative training on split data | llm | 0.85 |
| hardware | 8-core CPU with 16 threads | llm | 0.85 |
| hardware | 64-core machine (batch-latency sweep) | llm | 0.85 |
| security level | 128-bit | llm | 0.85 |
| threat model | outsourced training on an untrusted server; data confidentiality against malicious parties | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| limitations | About 2% accuracy loss for the MLP on the mortality dataset, attributed to quantization | llm | 0.85 |
| limitations | PBS error rate set to 1%; a 2^-40 error rate gives the same accuracy but is 2x slower | llm | 0.85 |
| limitations | Latency increases sharply for weight bit-widths above 6 bits | llm | 0.85 |
| limitations | Epoch latencies are long (up to 72.78 h for the MLP on mortality) | llm | 0.85 |
| llm relevance | medium | llm | 0.85 |
| key result | accuracy (FHE vs fp32): 98.25% vs 99.12% (logistic regression, breast-cancer, 6-bit; 11.8 s per batch, 0.23 h per epoch) | llm | 0.85 |
| key result | accuracy (FHE vs fp32): 90.09% vs 90.47% (logistic regression, mortality, 6-bit; 7.2 s per batch, 11.6 h per epoch) | llm | 0.85 |
| key result | accuracy (FHE vs fp32): 87.25% vs 90.44% (MLP (1 hidden), mortality, 4-bit; 45 s per batch, 72.78 h per epoch) | llm | 0.85 |
| key result | accuracy (FHE vs fp32): 88.52% vs 89.47% (logistic regression, heart, 6-bit; 8.02 s per batch, 0.08 h per epoch) | llm | 0.85 |
| key result | training throughput: 3 weight-gradient computations/s/thread (MLP on breast-cancer; prior works 24 [12] and 0.4 [13]) | llm | 0.85 |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: TFHE 14 [p.1], BGV 2 [p.2]
- library: Zama 5 [p.2], ONNX 2 [p.3], PyTorch 1 [p.3]
- model: Logistic regression 14 [p.2], MLP 13 [p.2]
- dataset: MNIST 1 [p.2]
- technique: Quantization 35 [p.2], Programmable bootstrapping 13 [p.3], Encrypted training 11 [p.1], Matrix multiplication 10 [p.3], Sigmoid 7 [p.2], Bootstrapping 3 [p.4], Lookup table 3 [p.3], ReLU 2 [p.2], Convolution 2 [p.2]
- task: Medical / healthcare 2 [p.7], Genomics 2 [p.7]
- topic: Homomorphic encryption 13 [p.2]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/ppml/training/montero-2024_machine-learning-training-on-encrypted-data-with-tfhe.pdf |
ppml/training | 7 | 1.2 MB | 5ae80d460d9a |
was Montero et al. - 2024 - Machine Learning Training on Encrypted Data with TFHE.pdf |
Database key: machine-learning-training-on-encrypted-data-with-tfhe · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set machine-learning-training-on-encrypted-data-with-tfhe COLUMN VALUE