Skip to content

FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System

Authors: Weizhao Jin; Yuhang Yao; Shanshan Han; Jiajun Gu; Carlee Joe-Wong; Srivatsan Ravi; Salman Avestimehr; Chaoyang He
Year: 2023
Venue: arXiv preprint

type: system collection: ppml status: auto

Links: arXiv:2303.10837

Summary

Auto-extracted summary (unreviewed)

FedML-HE is a federated-learning system with HE-based secure model aggregation whose main optimization, Selective Parameter Encryption, encrypts only the parameters a privacy-sensitivity map flags as most vulnerable to inversion attacks and leaves the rest in plaintext. Implemented over PALISADE and TenSEAL, it cuts HE computation and communication overhead by about 10x for ResNet-50 and up to about 40x for BERT; encrypting the top-10% sensitive parameters defeats DLG gradient inversion on CV models (vs 42.5% with random selection) and the top-30% defeats language-model inversion on BERT (vs 75% random).

Abstract

Federated Learning trains machine learning models on distributed devices by aggregating local model updates instead of local data. However, privacy concerns arise as the aggregated local models on the server may reveal sensitive personal information by inversion attacks. Privacy-preserving methods, such as homomorphic encryption (HE), then become necessary for FL training. Despite HE's privacy advantages, its applications suffer from impractical overheads, especially for foundation models. In this paper, we present FedML-HE, the first practical federated learning system with efficient HE-based secure model aggregation. FedML-HE proposes to selectively encrypt sensitive parameters, significantly reducing both computation and communication overheads during training while providing customizable privacy preservation. Our optimized system demonstrates considerable overhead reduction, particularly for large foundation models (e.g., ~10x reduction for ResNet-50, and up to ~40x reduction for BERT), demonstrating the potential for scalable HE-based FL deployment.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution FedML-HE: practical federated learning system with HE-based secure aggregation llm 0.80
contribution Selective Parameter Encryption guided by a parameter privacy-sensitivity map llm 0.80
contribution overhead evaluation on models up to Llama-2 7B llm 0.80
contribution defense evaluation against gradient inversion (DLG) and language-model inversion attacks llm 0.80
schemes CKKS llm 0.80
libraries PALISADE llm 0.80
libraries TenSEAL llm 0.80
libraries FedML llm 0.80
models Llama-2-7B llm 0.80
models BERT llm 0.80
models ResNet-50 llm 0.80
models ResNet-18 llm 0.80
models LeNet llm 0.80
models linear model llm 0.80
datasets CIFAR-100 llm 0.80
datasets WikiText llm 0.80
tasks federated learning llm 0.80
tasks secure model aggregation llm 0.80
tasks image classification llm 0.80
tasks language modeling llm 0.80
hardware Intel i7-7700 8-core 3.60 GHz CPU with 32 GB RAM llm 0.80
hardware NVIDIA Tesla T4 GPU llm 0.80
hardware AWS single-region bandwidth setting llm 0.80
security level 128-bit llm 0.80
threat model compromised or malicious server and eavesdroppers mounting data-reconstruction (gradient inversion) attacks on local model updates llm 0.80
interaction non-interactive FHE llm 0.80
code url https://github.com/IBM/federated-learning-lib/blob/main/ (URL found in the full text) regex 0.50
llm relevance medium llm 0.80
key result overhead reduction: about 10x (ResNet-50 with Selective Parameter Encryption) llm 0.80
key result overhead reduction: up to about 40x (BERT with Selective Parameter Encryption) llm 0.80
key result encrypted fraction needed to defeat DLG inversion: top-10% (sensitivity-selected) vs 42.5% (random) (CV model, CIFAR-100 samples) llm 0.80
key result encrypted fraction needed to defeat LM inversion: top-30% (selected) vs 75% (random) (BERT, WikiText samples) llm 0.80
key result overhead at 10% encryption: close to plaintext aggregation (HE computation and communication) llm 0.80

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Differential privacy 27 [p.1], Threshold HE 6 [p.4], Paillier 4 [p.2], CKKS 3 [p.15], MPC 3 [p.11], Multi-key HE 2 [p.12]
  • library: PALISADE 8 [p.8], TenSEAL 7 [p.8], Microsoft SEAL 6 [p.19], HELayers 4 [p.19], HElib 1 [p.19]
  • model: ResNet 16 [p.1], BERT 10 [p.1], Transformer 5 [p.12], LeNet 5 [p.5], CNN 5 [p.16], LLaMA 4 [p.8], RNN / LSTM 4 [p.2], ViT 3 [p.2], GAN 2 [p.9], MobileNet 1 [p.16], MLP 1 [p.16], GNN 1 [p.13]
  • dataset: CIFAR-100 2 [p.8]
  • hardware: GPU 2 [p.8], NVIDIA T4 1 [p.8], Raspberry Pi 1 [p.17]
  • technique: Federated learning 34 [p.1], Secure aggregation 12 [p.1], Semi-honest model 4 [p.3], Model inversion 2 [p.8]
  • task: IoT / edge 3 [p.2], Finance 2 [p.12]
  • topic: Homomorphic encryption 164 [p.1], Large language models 3 [p.1], Benchmark 3 [p.12], Open source 3 [p.12]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/ppml/federated-learning/jin-2023_fedml-he-efficient-he-based-privacy-preserving-federated-learning-system.pdf ppml/federated-learning 20 3.8 MB 30240fd56334 was Jin et al. - 2023 - FedML-HE An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System.pdf

Database key: fedml-he-an-efficient-homomorphic-encryption-based-privacy-preserving · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set fedml-he-an-efficient-homomorphic-encryption-based-privacy-preserving COLUMN VALUE