TL;DR

HCNN is the first GPU-accelerated homomorphic CNN that runs pre-trained inference on encrypted images under the BFV scheme, classifying MNIST in 5.16 s at 99% accuracy and CIFAR-10 in 304.43 s at 77.55% accuracy, with >80-bit security [Abstract][§I.A].

Problem and motivation

The work targets Deep Learning as a Service (DLaaS) where a cloud holds a pretrained model and a client wants predictions on sensitive inputs without leaking them to the server [§I]. The threat model is a non-interactive, single-round client/server setting: the client encrypts inputs and FHE evaluation keys, the server performs encrypted inference and returns the ciphertext, and “nothing about the input x is revealed to the cloud nor the model f to the client except what can be learned from input x and f(x)” [§I]. The paper does not explicitly name “honest-but-curious”, but the construction is consistent with a passive server — no integrity or malicious-client mechanisms are described. Prior pure-FHE work (CryptoNets) needed 570 s per MNIST evaluation on CPU; the authors argue GPUs can dramatically close this gap and bring DLaaS within practical reach [§I, p. 1331].

Key contributions

FHE setup

ML setup

Datasets

Dataset Task Size (train/test) Modality Notes
MNIST [§V.B] 10-class digit classification 50,000 / 10,000 Grayscale 28x28 images, pixel values 0–255 Inputs normalized to [0,1], scaled by 4, rounded to integers
CIFAR-10 [§V.B] 10-class image classification 50,000 / 10,000 Color 32x32x3 images, pixel values 0–255 Inputs normalized to [0,1], padding used; weights quantized to 8 bits

Pipeline diagram

flowchart LR
    subgraph Client
        A[Normalize and scale] --> B[Scalar encode]
        B --> C[Pack pixel i across images]
        C --> D[BFV encrypt with pk]
    end
    subgraph Server [Server GPU A*FV]
        E[HMultPlain Conv]
        E --> F[HSquare activation]
        F --> G[AvgPool optional]
        G --> H[HMultPlain FC]
    end
    subgraph Client_dec [Client]
        I[BFV decrypt with sk] --> J[CRT recombine]
        J --> K[Unpack and argmax]
    end
    D -. ciphertext .-> E
    H -. ciphertext .-> I

Pipeline steps (text)

  1. Client normalizes each image to [0,1] and applies a scalar encoding (multiply by D, round) [§III.A.1, §IV].
  2. Client packs the same pixel index across up to N images into one BFV plaintext slot vector [§III.A, Fig. 1].
  3. Client BFV-encrypts each pixel-vector plaintext under the public key; for CIFAR-10, the client also splits across r=10 plaintext-CRT channels [§IV.B, Table 6].
  4. Server (GPU) evaluates the convolutional layers as HMultPlain (encrypted input by plaintext weight) plus HAdd [§IV.C].
  5. Server applies square activation via HSquare on each ciphertext [§IV.C, Table 9].
  6. For CIFAR-10, server applies average pooling as HMultPlain with constant weights 1/n [§IV.C].
  7. Server evaluates remaining conv/FC layers, partitioning feature maps into GPU-memory-sized blocks where needed [§IV.D.1].
  8. Server returns the final ciphertext(s) to the client.
  9. Client BFV-decrypts; for CIFAR-10, applies CRT reconstruction over the 10 channels [§III.A.2, §IV.B].
  10. Client unpacks the slot vector to recover per-image logits and takes argmax [§III.A].

Architecture diagram

MNIST HCNN (5 layers)

flowchart LR
    I([Input 1x28x28]) --> C1[Conv 5, 5x5 s2]
    C1 --> A1[Square]
    A1 --> C2[Conv 50, 5x5 s2]
    C2 --> A2[Square]
    A2 --> F[Flatten]
    F --> FC1[FC 10]
    FC1 --> O([argmax])

CIFAR-10 HCNN (11 layers)

flowchart LR
    I([Input 3x32x32]) --> C1[Conv 32, 3x3]
    C1 --> A1[Square]
    A1 --> P1[AvgPool 2x2]
    P1 --> C2[Conv 64, 3x3]
    C2 --> A2[Square]
    A2 --> P2[AvgPool 2x2]
    P2 --> C3[Conv 128, 3x3]
    C3 --> A3[Square]
    A3 --> P3[AvgPool 2x2]
    P3 --> FC1[FC 256]
    FC1 --> FC2[FC 10]
    FC2 --> O([argmax])

Results

Headline numbers from [Table 10, Table 11, §V.D]:

Metric This paper Baseline Hardware
MNIST accuracy 99.00% (scalar encoding) CryptoNets 99.00%; E2DM 98.01%; FCryptoNets 98.71% N/A
MNIST inference (total) 5.16 s (A*FV, param set 1, 82-bit sec) / 5.71 s (param set 3, 175-bit sec) CryptoNets 570 s; E2DM 28.59 s; FCryptoNets 39.10 s V100 GPU vs. various CPUs
MNIST amortized per image 0.63 ms (set 1) / 0.34 ms (set 3) CryptoNets 69.58 ms; E2DM 450 ms; FCryptoNets 39.10 s V100 GPU
MNIST inference (SEAL CPU) 739.90 s (param set 2) / 1,563.85 s (param set 4) — 2x Intel Xeon Platinum (26 cores, 2.10 GHz, 187.5 GB DDR4)
MNIST GPU speedup vs SEAL 138.56x (set 2) / 258.92x (set 4) — V100 vs CPU
CIFAR-10 accuracy 77.55% (scalar encoding) / 77.80% (float) CryptoDL 91.50% (80-bit sec); FCryptoNets 75.99% N/A
CIFAR-10 inference (1 GPU) 553.89 s per CRT channel CryptoDL 11,686 s; FCryptoNets 22,372 s 1x V100
CIFAR-10 inference (4 GPUs) 304.43 s per CRT channel; amortized 0.372 s/image — 1x V100 + 3x P100 (assumes 10 machines run the 10 CRT channels in parallel)
Security level >80 bit (82–175 bit MNIST; 91 bit CIFAR-10) CryptoNets/E2DM/CryptoDL 80-bit —

Hardware testbed [Table 8, p. 1339]: CPU = 2x Intel Xeon Platinum, 26 cores, 2.10 GHz, 187.5 GB DDR4, 30 GB/s. GPU cluster = 1x V100 (5120 cores, 1.380 GHz, 16 GB HBM2, 732 GB/s) + 3x P100 (3584 cores, 1.328 GHz, 16 GB HBM2 each, 900 GB/s), connected by 16 GB/s PCIe. All single-GPU experiments used the V100 card.

Per-primitive GPU speedups over SEAL (param set 4) [Table 9]: 25.38x KeyGen, 17.37x Enc, 106.20x Dec, 5.20x HAdd, 58.29x HSquare, 448.40x HMultPlain, 62.54x HMult.

Limitations and assumptions

CryptoNets [Dowlin et al. 2016], FHE-DiNN [Bourse et al. 2018], E2DM [Jiang et al. 2018], MiniONN [Liu et al. 2017], GAZELLE [Juvekar et al. 2018], DELPHI [Srinivasan et al. 2019], XONN [Riazi et al. 2019], Faster CryptoNets (FCryptoNets) [Chou et al. 2018], CryptoDL [Hesamifard et al. 2017] [§I.B, Table 1, Table 11].

Code and artifacts

Not released. The paper does not provide a public repository for HCNN or the A*FV GPU library; the underlying SEAL library is open source [§IV].

Extra diagrams (optional)

Threat model

flowchart LR
    C[Client] -- "BFV enc(x) + evk" --> S[Cloud / Server]
    S -- "BFV enc(f(x))" --> C
    note[Server holds plaintext model f, learns nothing about x]

Activation approximation

The paper uses the square activation f(z) = z^2 as the polynomial replacement for ReLU [§III.B.1, §IV.C]. This is degree 2 with depth 1, doubling the scaling factor each application (D_out = D_in^2) [Table 7]. No new polynomial approximation is proposed; see CryptoNets [Dowlin et al. 2016] for prior use.

Open questions