FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System¶
Authors: Weizhao Jin; Yuhang Yao; Shanshan Han; Jiajun Gu; Carlee Joe-Wong; Srivatsan Ravi; Salman Avestimehr; Chaoyang He
Year: 2023
Venue: arXiv preprint
type: system collection: ppml status: auto
Links: arXiv:2303.10837
Summary¶
Auto-extracted summary (unreviewed)
FedML-HE is a federated-learning system with HE-based secure model aggregation whose main optimization, Selective Parameter Encryption, encrypts only the parameters a privacy-sensitivity map flags as most vulnerable to inversion attacks and leaves the rest in plaintext. Implemented over PALISADE and TenSEAL, it cuts HE computation and communication overhead by about 10x for ResNet-50 and up to about 40x for BERT; encrypting the top-10% sensitive parameters defeats DLG gradient inversion on CV models (vs 42.5% with random selection) and the top-30% defeats language-model inversion on BERT (vs 75% random).
Abstract¶
Federated Learning trains machine learning models on distributed devices by aggregating local model updates instead of local data. However, privacy concerns arise as the aggregated local models on the server may reveal sensitive personal information by inversion attacks. Privacy-preserving methods, such as homomorphic encryption (HE), then become necessary for FL training. Despite HE's privacy advantages, its applications suffer from impractical overheads, especially for foundation models. In this paper, we present FedML-HE, the first practical federated learning system with efficient HE-based secure model aggregation. FedML-HE proposes to selectively encrypt sensitive parameters, significantly reducing both computation and communication overheads during training while providing customizable privacy preservation. Our optimized system demonstrates considerable overhead reduction, particularly for large foundation models (e.g., ~10x reduction for ResNet-50, and up to ~40x reduction for BERT), demonstrating the potential for scalable HE-based FL deployment.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | FedML-HE: practical federated learning system with HE-based secure aggregation | llm | 0.80 |
| contribution | Selective Parameter Encryption guided by a parameter privacy-sensitivity map | llm | 0.80 |
| contribution | overhead evaluation on models up to Llama-2 7B | llm | 0.80 |
| contribution | defense evaluation against gradient inversion (DLG) and language-model inversion attacks | llm | 0.80 |
| schemes | CKKS | llm | 0.80 |
| libraries | PALISADE | llm | 0.80 |
| libraries | TenSEAL | llm | 0.80 |
| libraries | FedML | llm | 0.80 |
| models | Llama-2-7B | llm | 0.80 |
| models | BERT | llm | 0.80 |
| models | ResNet-50 | llm | 0.80 |
| models | ResNet-18 | llm | 0.80 |
| models | LeNet | llm | 0.80 |
| models | linear model | llm | 0.80 |
| datasets | CIFAR-100 | llm | 0.80 |
| datasets | WikiText | llm | 0.80 |
| tasks | federated learning | llm | 0.80 |
| tasks | secure model aggregation | llm | 0.80 |
| tasks | image classification | llm | 0.80 |
| tasks | language modeling | llm | 0.80 |
| hardware | Intel i7-7700 8-core 3.60 GHz CPU with 32 GB RAM | llm | 0.80 |
| hardware | NVIDIA Tesla T4 GPU | llm | 0.80 |
| hardware | AWS single-region bandwidth setting | llm | 0.80 |
| security level | 128-bit | llm | 0.80 |
| threat model | compromised or malicious server and eavesdroppers mounting data-reconstruction (gradient inversion) attacks on local model updates | llm | 0.80 |
| interaction | non-interactive FHE | llm | 0.80 |
| code url | https://github.com/IBM/federated-learning-lib/blob/main/ (URL found in the full text) | regex | 0.50 |
| llm relevance | medium | llm | 0.80 |
| key result | overhead reduction: about 10x (ResNet-50 with Selective Parameter Encryption) | llm | 0.80 |
| key result | overhead reduction: up to about 40x (BERT with Selective Parameter Encryption) | llm | 0.80 |
| key result | encrypted fraction needed to defeat DLG inversion: top-10% (sensitivity-selected) vs 42.5% (random) (CV model, CIFAR-100 samples) | llm | 0.80 |
| key result | encrypted fraction needed to defeat LM inversion: top-30% (selected) vs 75% (random) (BERT, WikiText samples) | llm | 0.80 |
| key result | overhead at 10% encryption: close to plaintext aggregation (HE computation and communication) | llm | 0.80 |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: Differential privacy 27 [p.1], Threshold HE 6 [p.4], Paillier 4 [p.2], CKKS 3 [p.15], MPC 3 [p.11], Multi-key HE 2 [p.12]
- library: PALISADE 8 [p.8], TenSEAL 7 [p.8], Microsoft SEAL 6 [p.19], HELayers 4 [p.19], HElib 1 [p.19]
- model: ResNet 16 [p.1], BERT 10 [p.1], Transformer 5 [p.12], LeNet 5 [p.5], CNN 5 [p.16], LLaMA 4 [p.8], RNN / LSTM 4 [p.2], ViT 3 [p.2], GAN 2 [p.9], MobileNet 1 [p.16], MLP 1 [p.16], GNN 1 [p.13]
- dataset: CIFAR-100 2 [p.8]
- hardware: GPU 2 [p.8], NVIDIA T4 1 [p.8], Raspberry Pi 1 [p.17]
- technique: Federated learning 34 [p.1], Secure aggregation 12 [p.1], Semi-honest model 4 [p.3], Model inversion 2 [p.8]
- task: IoT / edge 3 [p.2], Finance 2 [p.12]
- topic: Homomorphic encryption 164 [p.1], Large language models 3 [p.1], Benchmark 3 [p.12], Open source 3 [p.12]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/ppml/federated-learning/jin-2023_fedml-he-efficient-he-based-privacy-preserving-federated-learning-system.pdf |
ppml/federated-learning | 20 | 3.8 MB | 30240fd56334 |
was Jin et al. - 2023 - FedML-HE An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System.pdf |
Database key: fedml-he-an-efficient-homomorphic-encryption-based-privacy-preserving · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set fedml-he-an-efficient-homomorphic-encryption-based-privacy-preserving COLUMN VALUE