TL;DR

FedML-HE is the first practical CKKS-based homomorphic-encryption federated learning system that, by encrypting only the most privacy-sensitive parameters (“Selective Parameter Encryption”), cuts HE overhead roughly 10x for ResNet-50 and up to 40x for BERT while keeping exact aggregation [Abstract][§1].

Problem and motivation

Standard federated learning leaks information because plaintext local model updates received by the aggregation server can be inverted to reconstruct training data [§1, Fig.1]. Differential privacy degrades accuracy and secure aggregation needs synchronization and is fragile to client dropout [Table 1]. HE-FL preserves exact gradients and is dropout-resilient but introduces ~15x computation/communication overhead, making it infeasible for foundation models [§1].

Threat model: a semi-honest adversary A that may corrupt either the aggregation server or any subset of clients (but not both simultaneously, in the single-key setup) [Definition 3.1, §2.2]. A threshold variant covers server+up to n-k client corruption [Definition 3.2]. No collusion is assumed between the trusted key authority and the aggregation server [§B].

Key contributions

FHE setup

ML setup

Datasets

Dataset Task Size (train/test) Modality Notes
CIFAR-100 Image classification (used to compute LeNet sensitivity & for DLG attack evaluation) Not reported Images Used for parameter sensitivity map and gradient-inversion attack evaluation [§4.2.2].
wikitext Language modeling (BERT inversion attack evaluation) Not reported Text Used to evaluate Language Model Inversion Attacks defense [§4.2.2, Fig.10].

(System overheads are benchmarked on models without specifying downstream datasets [Table 4].)

Pipeline diagram

flowchart LR
    subgraph KeyAuth [Key Authority]
        K[KeyGen pk,sk]
    end
    subgraph Clients
        L[Local train Wi] --> S[Sensitivity map Si]
        S --> ES[Enc Si]
        L --> EM[Enc selected Wi]
    end
    subgraph Server
        AGG[HE aggregate Si]
        MASK[Build mask M, ratio p]
        FAGG[HE aggregate weighted Wi]
    end
    K -. pk,sk .-> Clients
    K -. crypto ctx .-> Server
    ES -. ciphertext .-> AGG
    AGG --> MASK
    MASK -. enc M .-> Clients
    EM -. partial ct .-> FAGG
    FAGG -. enc global .-> Clients

Pipeline steps (text)

  1. Key Authority runs HE.KeyGen(lambda) and distributes (pk, sk) to clients and crypto context to the server [Algorithm 1, §2.3].
  2. Each client computes a local sensitivity map Si over its dataset Di using gradient sensitivity w.r.t. input [§2.4 Step 1].
  3. Clients encrypt Si and send [[Si]] to the server [Algorithm 1].
  4. Server homomorphically aggregates the sensitivity maps into a global privacy map and selects the top-p ratio of parameters as the encryption mask M [§2.4 Step 2].
  5. Clients decrypt M (round 1) and use it to selectively encrypt the most sensitive parameters of their trained model Wi; the rest is sent in plaintext [Algorithm 1].
  6. Server aggregates: [Wglob] = sum_i alpha_i [[M (X) Wi]] + sum_i alpha_i ((1-M) (X) Wi) [§2.3 equation].
  7. Clients receive [Wglob], decrypt the encrypted portion using sk, recombine with the plaintext portion, and proceed to the next training round [Algorithm 1].

Architecture diagram

FedML-HE is a federated training system, not a single-model FHE inference pipeline. The diagram below shows the framework’s three-layer software stack [§2.5, Fig.6].

flowchart LR
    subgraph FLOrch [FL Orchestration]
        SM[Server Manager] --- SA[Server Aggregator]
        CM[Client Manager] --- CT[Client Trainer]
    end
    subgraph MLBridge [ML Bridge]
        MP[ML Processing] --> MF[Model Flatten]
        MF --> MR[Model Reshape]
        OPT[Selective Param Encryption]
    end
    subgraph CryptoFnd [Crypto Foundation]
        KG[KeyGen]
        ED[Enc / Dec]
        HA[HE Agg Functions]
        CP[Ciphertext Packing]
        HL[(HE Libs: PALISADE, TenSEAL)]
    end
    FLOrch --> MLBridge
    MLBridge --> CryptoFnd
    CryptoFnd --> HL

Results

Headline overhead reductions and accuracy.

Metric This paper Baseline Hardware
ResNet-50 HE-FL time (1 round, 3 clients) 46.672 s (vanilla full-encrypt); ~10x reduction via selective encryption (Abstract, §1) Plaintext 5.379 s [Table 4] Intel i7-7700 8-core 3.60GHz, 32 GB RAM, Tesla T4, Ubuntu 18.04.6 [§4.1]
BERT HE-FL time 136.914 s (vanilla full-encrypt); up to ~40x reduction via selective + parameter-efficiency (Abstract) Plaintext 19.674 s [Table 4] Same as above
Llama-2 (7B) HE-FL time 13067.154 s Plaintext 2423.976 s (5.39x ratio) [Table 4] Same as above
ViT @ 10% selective encryption 30.874 s, 844.49 MB Full encryption 112.504 s, 5.35 GB [Table 7] Same as above
CNN (2 Conv + 2 FC) model-test accuracy delta -1.85% to +0.32% across packing/scaling configs [Table 6] Plaintext baseline Same as above
Defense vs DLG on LeNet/CIFAR-100 Top-10% sensitive encryption suffices [§4.2.2] Random selection needs 42.5% [§4.2.2] Same as above
Defense vs LM inversion on BERT/wikitext 30% selective beats 75% random (accuracy 0.0820 vs 0.1973) [Fig.10] — Same as above
FedML-HE (PALISADE) vs NVIDIA FLARE (CNN, 3 clients) 2.456 s, 105.72 MB (vanilla); 0.874 s, 16.37 MB with opt FLARE TenSEAL 2.826 s, 129.75 MB; IBMFL 3.955 s, 86.58 MB [Table 8] Same as above

Limitations and assumptions

Code and artifacts

Not released (no explicit repository link given in the paper text). FedML Inc. is the corresponding institution and the paper describes a “Deploy Anywhere” platform [§C.2], but no public URL is provided in the .txt.

Extra diagrams (optional)

Threat model

flowchart LR
    KA[Key Authority] -- pk,sk --> C[Clients]
    KA -- crypto ctx --> S[Aggregation Server]
    C -- enc selected Wi + plain rest --> S
    S -- enc global model --> C
    A((Semi-honest A)) -. corrupts .-> S
    A -. or corrupts .-> C

A may corrupt the server or any subset of clients, but not both simultaneously under the single-key setup [Def. 3.1]. No collusion between KA and S [§B].

Federated round

sequenceDiagram
    participant C as Clients
    participant S as Server
    C->>C: Local train Wi
    C->>S: enc(M (X) Wi) + plain((1-M) (X) Wi)
    S->>S: HE-weighted sum across clients
    S->>C: [Wglob]
    C->>C: Dec(sk, enc part) + plain part

Open questions