TL;DR

The authors give the first implementation of the standard ResNet-20 on the CIFAR-10 dataset entirely under the RNS-CKKS FHE scheme with bootstrapping, using high-precision minimax polynomial approximations of ReLU and Softmax, reaching 92.43%+/-2.65% accuracy (98.43% identical to plaintext) in about 3 hours per image [Abstract][§I.A][§VI].

Problem and motivation

Prior FHE-based PPML systems (CryptoNets, SEALion, CryptoDL, etc.) replaced non-arithmetic activations like ReLU with simple polynomials and avoided bootstrapping, so they could not run deep, standard networks and were not accurate on benchmarks beyond MNIST [Abstract][§I][§I.B.1]. The closest prior deep system, Lou and Jiang’s leveled-TFHE ResNet-20, required impractically large parameters and could not exploit packing efficiently [§I][§I.B.3]. The threat model is a server-side PPML service: a client sends an FHE-encrypted image and public/evaluation keys, the server runs encrypted inference and returns the ciphertext, and the model parameters stay plaintext on the server [§I]. The paper also explicitly mitigates the model-extraction attack of Boemer et al. by evaluating Softmax under FHE rather than returning pre-Softmax logits to the client [§I.A].

Key contributions

FHE setup

ML setup

Datasets

Dataset Task Size (train/test) Modality Notes
CIFAR-10 10-class image classification Standard split; tested on 10,000 plaintext images; agreement/accuracy under encryption measured on 383 encrypted images with 95% CIs 32x32x3 RGB images Mean-subtracted, shift + horizontal-mirror augmentation during plaintext training [§V.A][§VI.A][§VI.B]

Pipeline diagram

flowchart LR
    subgraph Client
        A[CIFAR-10 image] --> B[Mean subtract]
        B --> C[Encode + Encrypt CKKS]
    end
    subgraph Server
        D[Conv + BN]
        D --> E[Bootstrap]
        E --> F[ReLU minimax]
        F --> G[Residual blocks x3 stages]
        G --> H[AvgPool 8x8]
        H --> I[FC 64 to 10]
        I --> J[Softmax minimax]
    end
    subgraph Client_dec [Client]
        K[Decrypt with sk] --> L[Argmax label]
    end
    C -. ciphertext .-> D
    J -. ciphertext .-> K

Pipeline steps (text)

  1. Client preprocesses the 32x32x3 image, subtracts the training-set pixel mean [§VI.A].
  2. Client packs each channel into one ciphertext using 2^10 sparse slots and encrypts under RNS-CKKS [§V.B.2, Algorithm 3].
  3. Client uploads ciphertexts plus evaluation keys to the server [§I].
  4. Server applies the first Conv + BN (homomorphic addition + scalar multiplication) [§V.C, Algorithm 4].
  5. Server bootstraps immediately after each Conv to refresh levels (placement saves 27.8% vs after-ReLU) [§IV][§VI.B, Table 6].
  6. Server evaluates ReLU as 0.5 x (1 + sign(x)) using the deg-7/15/27 minimax composite polynomials, with two extra bootstraps per ReLU layer [§V.D, Algorithm 6].
  7. Server iterates Conv-BN-Boot-ReLU through three residual stages (16/32/64 channels), using stride-2 convolutions with window-filter masking at stage transitions [§III.B][§V.C].
  8. Server performs average pooling over each 8x8 channel via log-step rotation-adds, producing a length-64 vector across ciphertexts [§V.F, Algorithm 7].
  9. Server applies the FC 64 -> 10 layer (no rotations needed because pooling output is already separated across ciphertexts) [§V.F].
  10. Server evaluates Softmax: divide by B=64, apply deg-12 exp approximation, square log(B/lambda) times, Gumbel-Softmax with lambda=4, then 16 Goldschmidt division steps for the inverse, with bootstrapping inserted before Softmax, before the inverse, and after 8 Goldschmidt iterations [§V.G, Algorithm 8].
  11. Server returns the encrypted one-hot vector; client decrypts and reads the argmax label [§V.G][§I].

Architecture diagram

flowchart LR
    I([Input 3x32x32]) --> C0[Conv 16, 3x3, s1]
    C0 --> A0[BN + minimax ReLU]
    A0 --> S1[Stage 1: 3 blocks, 16ch, 32x32]
    S1 --> S2[Stage 2: 3 blocks, 32ch, 16x16, stride 2]
    S2 --> S3[Stage 3: 3 blocks, 64ch, 8x8, stride 2]
    S3 --> AP[AvgPool 8x8]
    AP --> FC[FC 64 to 10]
    FC --> SM[Softmax minimax]
    SM --> O([Encrypted one-hot])

Results

Metric This paper Baseline Hardware
Encrypted-data classification accuracy on CIFAR-10 92.43% +/- 2.65% (95% CI, 383 encrypted images) Plaintext ResNet-20 on same model: 92.95% +/- 2.56%; original published ResNet-20: 91.89% Dual Intel Xeon Platinum 8280, 112 cores, 172 GB RAM [§VI.A-B]
Agreement ratio with plaintext ResNet-20 98.43% +/- 1.25% n/a Same [§VI.B, Table 4]
End-to-end inference latency (single image) “about 3 h” per image (~10,800 s) n/a (no prior FHE ResNet-20 with bootstrapping) Dual Xeon Platinum 8280 (112 cores), one thread per channel via OpenMP, 172 GB RAM [§VI.A][§VI.B, Table 5]
Bootstrapping placement 27.8% runtime reduction when Boot is after Conv rather than after ReLU After-ReLU baseline Same [§VI.B, Table 6]
Best HE-friendly CNN baseline on CIFAR-10 (prior word-wise FHE work) n/a 91.5% accuracy [20] Different setups [§I.B.1]
Security level 111.6-bit (hybrid dual attack [29]) 128-bit standard target n/a [§V.B.1][§VI.C.2]

Limitations and assumptions

Code and artifacts

Not released — the paper does not link a repository. The implementation is described as built on top of Microsoft SEAL 3.6.1 with the authors’ own bootstrapping module added [§I.A][§V.A].

Extra diagrams (optional)

Threat model

flowchart LR
    C[Client] -- enc(image) + eval keys --> S[Server with plaintext ResNet-20 weights]
    S -- enc(one-hot via Softmax) --> C

Activation approximation

ReLU(x) = 0.5 * x * (1 + sign(x)). sign(x) is the composition p2 o p1 o p0 of three minimax polynomials of degree 7, 15, 27 over symmetric intervals around 0, with alpha=12 (i.e., precision 2^-12 on x in [2^-12, 1]). Average precision is ~16 bits; bootstrapping is invoked twice per ReLU layer [§V.D, Algorithms 5-6]. See also Section II-D and reference [14].

Open questions