TL;DR

The paper accelerates exact (decryption-error-free) FHE-encrypted CNN/DNN inference over NuFHE (a GPU TFHE port) by combining INT8 quantization with software bitslicing of bootstrapped Boolean gates, packing many parallel bits into single NuFHE gate calls [§I, §IV]. Reported speedup is ~37x over the same non-bitsliced NuFHE baseline, while keeping memory at ~700 MB for a CNN and ~1.9 GB for a DNN on MNIST [§I, §V, Fig. 8].

Problem and motivation

ML-as-a-Service workloads on public clouds (AWS, Azure, GCP) expose private inputs to untrusted servers; the authors cite recent vulnerabilities in those providers to motivate encrypted analytics [§I, refs 1-3]. Existing FHE-for-ML options each fail one criterion: LHE/SWHE limit depth and force polynomial/square activations; CKKS has known key-recovery attacks on its approximate decryption [§II, refs 12-13]; Zama Concrete (TFHE-based programmable bootstrapping) is shown experimentally to incur decryption errors on integer plaintexts during leveled/FHE compositions [§II, Tables II-III]. The authors keep an exact Boolean-gate TFHE library (NuFHE) intact and instead attack the cost via architecture tricks [§II]. Threat model is honest-but-curious server, public-model / private-data (MLaaS) [§I, “public model private data assumption”].

Key contributions

FHE setup

ML setup

Datasets

Dataset Task Size (train/test) Modality Notes
MNIST [§IV-A, ref 35] Handwritten-digit classification (10 classes) 60,000 images (28x28), standard split Greyscale images INT8 quantized via TFLite; primary timing/memory benchmark [§V, Tables VII-VIII]
Mendeley Concrete Crack [§IV-A, ref 36] Binary classification (crack / no-crack) 20K per class, 224x224x3 RGB images of concrete surfaces Used to stress packing-width limit (224/32 = 7 packings per row) and to show INT8 sometimes raises accuracy [§IV-A, Table VI; §V, Table IX]

Pipeline diagram

flowchart LR
    subgraph Client
        A[Input image] --> B[INT8 quantize]
        B --> C[Bit-decompose + PACK]
        C --> D[NuFHE encrypt bits]
    end
    subgraph Server_GPU [Server NuFHE GPU]
        E[Bitsliced Conv2D] --> F[FHE-MUX ReLU]
        F --> G[AvgPool2D bit-shift]
        G --> H[Bitsliced Conv2D]
        H --> I[FHE-MUX ReLU]
        I --> J[AvgPool2D bit-shift]
        J --> K[Flatten + Dense BFA]
        K --> L[Dense logits ct]
    end
    subgraph Client_dec [Client]
        M[NuFHE decrypt] --> N[Unpack + argmax]
    end
    D -. ciphertext bits .-> E
    L -. ciphertext bits .-> M

Pipeline steps (text)

  1. Train plaintext CNN in TFLite and post-training quantize weights and activations to INT8 [§IV-A, ref 33].
  2. Client bit-decomposes each INT8 input pixel and groups bits across N parallel lanes using PACK(j) into k=8 registers [§IV-B-1, Algorithm 1].
  3. Client encrypts each packed bit-register with NuFHE (TFHE secret key) and ships ciphertexts to the GPU server [§III, §V].
  4. Server runs bitsliced Conv2D: for each weight w_ij, repeat-add the packed input vector P_{w_ij} using the BFA (w_ij - 1) BFA invocations, then sum product terms via another BFA pass [§IV-B-2, Fig. 5].
  5. Server applies FHE-MUX-based ReLU using the MSB of an FHE subtraction as selector between encrypted 0 and encrypted x [§IV-B-2].
  6. Server performs AvgPool2D as bit-shift-right-by-2 on the four input ciphertexts (i.e., divide-by-4) [§IV-B-2, Fig. 6].
  7. Repeat Conv→ReLU→AvgPool for the second block, then flatten, then bitsliced Dense (group ciphertexts by shared plaintext weight, BFA-add) [§IV-B-2].
  8. Server returns final ciphertext logit register to client.
  9. Client decrypts with NuFHE secret key, unpacks bits to INT8 logits, takes argmax [§III].

Architecture diagram

flowchart LR
    I([Input 1x28x28 MNIST]) --> C1[Conv2D 3x3]
    C1 --> A1[Exact ReLU FHE-MUX]
    A1 --> P1[AvgPool2D 2x2 bit-shift]
    P1 --> C2[Conv2D 3x3]
    C2 --> A2[Exact ReLU FHE-MUX]
    A2 --> P2[AvgPool2D 2x2 bit-shift]
    P2 --> F[Flatten]
    F --> FC1[Dense]
    FC1 --> A3[Exact ReLU FHE-MUX]
    A3 --> FC2[Dense logits]
    FC2 --> O([Argmax after decrypt])

Results

Plaintext baselines: INT8-quantized models give ~98% on MNIST and a slight gain on Mendeley vs FP32 (≤2% loss reported as typical) [§IV-A, Table VI]. Bitslicing matches the non-bitsliced NuFHE encrypted accuracy exactly because no approximation is introduced [§II, §IV-B].

Metric This paper Baseline Hardware
MNIST CNN end-to-end encrypted inference (single image) ~4.5 h [§V, Table VIII] ~80 h non-bitsliced NuFHE CNN, same arch [§I, §IV-B] Ubuntu 18.04, 256 GB RAM, 24 GB GPU (NuFHE) [§V]
MNIST DNN (2x50 + 10) encrypted inference 2.15 h [§V, Table VII] ~80 h non-bitsliced [§V] Same
Mendeley 224x224x3 CNN inference (estimated) ~140 h [§V, Table IX] Not reported Same
Speedup over non-bitsliced NuFHE ~37x [§I, §V] 1x Same
Peak memory, CNN inference ~700 MB [§I, §V, Fig. 8a] 6.7 GB non-bitsliced NuFHE baseline [§I] Same
Peak memory, DNN inference ~1.9 GB [§V, Fig. 8b] Not reported Same
BFA adder memory 280 MB 5.2 GB naive Same [§IV-B-1]
Single Conv2D memory (vs SHE [7]) Bitsliced Conv2D ~180 MB; 50 convolutions ~9 GB for SHE multithreaded baseline CryptoNets [4] 2-3 GB on MNIST; SHE [7] few GB Same [§V, Table X, Fig. 9]
Accuracy (MNIST, INT8 quantized) ~98% (≤2% drop vs FP32) FP32 plaintext baseline [§IV-A, Table VI] Plaintext eval

The ~4.5 h single-image MNIST CNN latency (16,200 s) is what populates single_inference_seconds. No CIFAR-10 / ImageNet numbers are reported; Mendeley CNN is “estimated”, not measured end-to-end [§V].

Limitations and assumptions

Code and artifacts

Not released. The paper provides no repository for its NuFHE wrapper or bitsliced building blocks. The only external code link cited is the SHE baseline at https://github.com/qianlou/SHE [§V, footnote 1]. NuFHE itself (the underlying library) is at https://nufhe.readthedocs.io [§V, ref 16].

Extra diagrams (optional)

Threat model

flowchart LR
    C[Client owns sk, INT8 data] -- packed NuFHE ciphertexts --> S[Cloud GPU server, public model]
    S -- encrypted logits --> C

Open questions