Skip to content

Zephyr: GPU-Efficient Homomorphic Encryption for Privacy-Preserving Transformer Inference

Authors: Seo, Min
Year: 2026
Venue: IACR ePrint 2026/932

type: system collection: catalogue, schemes status: auto

Links: ePrint 2026/932

Summary

Auto-extracted summary (unreviewed)

Zephyr is a GPU-oriented CKKS framework for private transformer inference that builds the RNS basis entirely from 30-bit primes and uses a grafting-based representation (a 30-graft system) to decouple scale management from the modulus chain, so that the scheme runs on efficient 32-bit GPU integer arithmetic while retaining flexible rescaling. It also optimizes ciphertext-ciphertext matrix multiplication for the Q.K^T attention product by removing redundant linear transformations and merging overlapping rotation patterns. End-to-end encrypted BERT inference on RTE takes 107.94 s on an A100, 5.77x faster than Thor and 3.18x faster than Powerformer, and the whole pipeline fits in a 24 GB RTX 4090, where it runs about 1.65x faster than on the A100.

Abstract

Privacy-preserving machine learning (PPML) enables inference over sensitive data without exposing raw inputs, with CKKS being a widely adopted scheme for approximate arithmetic. However, existing CKKS implementations are primarily based on 64-bit residue number system (RNS) representations, creating a mismatch with modern GPUs optimized for 32-bit integer arithmetic. This mismatch introduces substantial computational overhead, limiting the practicality of encrypted transformer inference. In this work, we present Zephyr, a GPU-efficient framework for homomorphic transformer inference via 32-bit arithmetic and grafting. Zephyr revisits the design of CKKS under GPU constraints and introduces a grafting-based representation that decouples scale management from the modulus chain. By constructing the RNS basis entirely with 30-bit primes and managing scale through auxiliary graft structures, Zephyr enables flexible rescaling while remaining compatible with efficient 32-bit GPU execution. Compared to Cheddar (Choi et al., ASPLOS’26), a representative GPU-oriented CKKS design based on fixed 25-30 prime systems, our approach simplifies modulus management and enables more flexible operations across different levels, while reducing rescaling overhead at the cost of additional convolution overhead. We further optimize ciphertext-ciphertext matrix multiplication (CCMM), a major bottleneck in encrypted transformer inference, by eliminating redundant linear transformations and merging overlapping rotation patterns in attention computation. Our theoretical and empirical analysis demonstrates that grafting-based 32-bit CKKS provides a practical and flexible design point for GPU-accelerated PPML inference.
Source: pdf

Extracted details

Field Value Source Confidence
contribution grafting-based 32-bit CKKS design (30-graft system) whose RNS basis consists only of 30-bit primes, decoupling scale management from the modulus chain llm 0.85
contribution optimized ciphertext-ciphertext matrix multiplication for attention (Q.K^T) that eliminates redundant linear transformations and merges overlapping rotation patterns llm 0.85
contribution BS-GS configurations that minimize switching keys under a 24 GB GPU memory budget llm 0.85
contribution theoretical and empirical comparison with Cheddar, Thor, Powerformer and Bolt on end-to-end BERT inference llm 0.85
schemes CKKS llm 0.85
models BERT llm 0.85
datasets RTE llm 0.85
tasks private transformer inference llm 0.85
tasks text entailment classification (RTE) llm 0.85
hardware NVIDIA A100 80GB GPU llm 0.85
hardware NVIDIA RTX 4090 24GB GPU llm 0.85
threat model outsourced cloud inference on encrypted inputs llm 0.85
interaction non-interactive FHE llm 0.85
code url https://github.com/tuneinsight/lattigo (URL found in the full text) regex 0.50
limitations memory-bound operations benefit less from reduced arithmetic precision, so the observed 3.18x is below the theoretical 4.76x llm 0.85
limitations grafting reduces rescaling overhead at the cost of additional convolution overhead llm 0.85
limitations minimizing switching keys increases the number of key-switching operations for CPMM llm 0.85
limitations speed-up comparisons are influenced by implementation differences across systems llm 0.85
llm relevance high llm 0.85
key result end-to-end inference latency: 107.94 s (BERT, RTE benchmark, NVIDIA A100 80GB) llm 0.85
key result speed-up over Thor: 5.77x (623.39 s -> 107.94 s) (end-to-end BERT, RTE, A100) llm 0.85
key result speed-up over Powerformer: 3.18x (344.32 s -> 107.94 s) (end-to-end BERT, RTE, A100) llm 0.85
key result bootstrapping time: 36.54 s (vs 337.86 s Thor, 103.72 s Powerformer) (end-to-end BERT, A100) llm 0.85
key result RTX 4090 vs A100 speed-up: about 1.65x (end-to-end inference within 24 GB GPU memory) llm 0.85
key result expected speed-up of 30-graft system: about 4.76x (theoretical) (under an identical-implementation assumption vs 64-bit RNS) llm 0.85

System card: Zephyr

Group 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK)
Family CKKS with 32-bit RNS + grafting
Arithmetic Approximate real (CKKS)
Interaction Non-interactive FHE
Model scale Transformer (see paper)
Status Active SOTA
Relation to the SoK Redesigns CKKS for 32-bit GPU integer arithmetic.
Note Combines 32-bit words with grafting to remove the 64-bit RNS mismatch on GPUs.
Systems page Zephyr

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 38 [p.1], MPC 4 [p.1]
  • library: Cheddar 6 [p.1], Lattigo 3 [p.3]
  • model: Transformer 7 [p.1], BERT 3 [p.8]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 1 [p.8]
  • hardware: GPU 39 [p.1], NVIDIA A100 13 [p.8], NVIDIA RTX 10 [p.8], Multi-GPU 1 [p.11]
  • technique: RNS 47 [p.1], Bootstrapping 19 [p.3], Rescaling 15 [p.1], Matrix multiplication 14 [p.1], Key switching 13 [p.2], NTT 13 [p.4], Attention 11 [p.3], Convolution 11 [p.1], LayerNorm 7 [p.3], Rotations 7 [p.1], Softmax 4 [p.3], Non-interactive 4 [p.1], GELU 3 [p.3]
  • topic: Homomorphic encryption 29 [p.1], Large language models 6 [p.6], Private / secure inference 5 [p.1], Privacy-preserving machine learning 4 [p.1], Benchmark 3 [p.8], Open source 2 [p.10]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/1-124_zephyr-gpu-efficient-homomorphic-encryption-for-privacy-pres.pdf catalogue 18 391.4 KB d6c7bc04e127
references/schemes/s07-1_eprint-2026-932.pdf schemes 18 391.4 KB d6c7bc04e127 byte-identical to references/catalogue/1-124_zephyr-gpu-efficient-homomorphic-encryption-for-privacy-pres.pdf

Database key: zephyr-gpu-efficient-homomorphic-encryption-for-privacy-preserving · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set zephyr-gpu-efficient-homomorphic-encryption-for-privacy-preserving COLUMN VALUE