Skip to content

Machine Learning Training on Encrypted Data with TFHE

Authors: Luis Montero; Jordan Frery; Celia Kherfallah; Roman Bredehoft; Andrei Stoian
Year: 2024
Venue: Proceedings of the 10th ACM International Workshop on Security and Privacy Analytics

type: technique collection: ppml status: auto

Links: DOI

Summary

Auto-extracted summary (unreviewed)

This short paper from Zama presents a TFHE-based framework for outsourcing the training of quantized machine-learning models on encrypted data, using programmable bootstrapping (including a rounded PBS trick) and compiled crypto-system parameters, aimed at collaborative settings with horizontally or vertically split data. Training logistic regression and one-hidden-layer MLPs on breast-cancer, mortality and heart datasets reaches accuracy within about 2% of fp32 plaintext training (e.g., 98.25% vs 99.12% for logistic regression on breast-cancer), with batch latencies of 7-149 s and epoch times from 0.08 h to 72.78 h, and a throughput of 3 weight-gradient computations per second per thread versus 0.4 for one prior work.

Abstract

We present an approach for outsourcing the training of machine learning (ML) models while preserving data confidentiality from malicious parties. We use fully homomorphic encryption (FHE) to build a unified training framework that works on encrypted data and learns quantized ML models. Our approach finds future applications in collaborative settings involving multiple parties working on confidential data, which can be horizontally or vertically split between data owners. We train logistic regression and multilayer perceptrons on several datasets and show results that are comparable to the state-of-the-art.
Source: pdf

Extracted details

Field Value Source Confidence
contribution Unified TFHE training framework for quantized logistic regression and MLPs on encrypted data llm 0.85
contribution Rounded programmable bootstrapping to drop least-significant bits cheaply llm 0.85
contribution Compilation-based selection of crypto-system parameters at 128-bit security llm 0.85
contribution Study of weight bit-width and quantization scaling trade-offs between accuracy and latency llm 0.85
schemes TFHE llm 0.85
models logistic regression llm 0.85
models MLP (1 hidden layer) llm 0.85
datasets breast cancer (OpenML) llm 0.85
datasets mortality llm 0.85
datasets heart llm 0.85
tasks encrypted ML training llm 0.85
tasks binary classification llm 0.85
tasks collaborative training on split data llm 0.85
hardware 8-core CPU with 16 threads llm 0.85
hardware 64-core machine (batch-latency sweep) llm 0.85
security level 128-bit llm 0.85
threat model outsourced training on an untrusted server; data confidentiality against malicious parties llm 0.85
interaction non-interactive FHE llm 0.85
limitations About 2% accuracy loss for the MLP on the mortality dataset, attributed to quantization llm 0.85
limitations PBS error rate set to 1%; a 2^-40 error rate gives the same accuracy but is 2x slower llm 0.85
limitations Latency increases sharply for weight bit-widths above 6 bits llm 0.85
limitations Epoch latencies are long (up to 72.78 h for the MLP on mortality) llm 0.85
llm relevance medium llm 0.85
key result accuracy (FHE vs fp32): 98.25% vs 99.12% (logistic regression, breast-cancer, 6-bit; 11.8 s per batch, 0.23 h per epoch) llm 0.85
key result accuracy (FHE vs fp32): 90.09% vs 90.47% (logistic regression, mortality, 6-bit; 7.2 s per batch, 11.6 h per epoch) llm 0.85
key result accuracy (FHE vs fp32): 87.25% vs 90.44% (MLP (1 hidden), mortality, 4-bit; 45 s per batch, 72.78 h per epoch) llm 0.85
key result accuracy (FHE vs fp32): 88.52% vs 89.47% (logistic regression, heart, 6-bit; 8.02 s per batch, 0.08 h per epoch) llm 0.85
key result training throughput: 3 weight-gradient computations/s/thread (MLP on breast-cancer; prior works 24 [12] and 0.4 [13]) llm 0.85

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: TFHE 14 [p.1], BGV 2 [p.2]
  • library: Zama 5 [p.2], ONNX 2 [p.3], PyTorch 1 [p.3]
  • model: Logistic regression 14 [p.2], MLP 13 [p.2]
  • dataset: MNIST 1 [p.2]
  • technique: Quantization 35 [p.2], Programmable bootstrapping 13 [p.3], Encrypted training 11 [p.1], Matrix multiplication 10 [p.3], Sigmoid 7 [p.2], Bootstrapping 3 [p.4], Lookup table 3 [p.3], ReLU 2 [p.2], Convolution 2 [p.2]
  • task: Medical / healthcare 2 [p.7], Genomics 2 [p.7]
  • topic: Homomorphic encryption 13 [p.2]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/ppml/training/montero-2024_machine-learning-training-on-encrypted-data-with-tfhe.pdf ppml/training 7 1.2 MB 5ae80d460d9a was Montero et al. - 2024 - Machine Learning Training on Encrypted Data with TFHE.pdf

Database key: machine-learning-training-on-encrypted-data-with-tfhe · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set machine-learning-training-on-encrypted-data-with-tfhe COLUMN VALUE