Zephyr: GPU-Efficient Homomorphic Encryption for Privacy-Preserving Transformer Inference¶
Authors: Seo, Min
Year: 2026
Venue: IACR ePrint 2026/932
type: system collection: catalogue, schemes status: auto
Links: ePrint 2026/932
Summary¶
Auto-extracted summary (unreviewed)
Zephyr is a GPU-oriented CKKS framework for private transformer inference that builds the RNS basis entirely from 30-bit primes and uses a grafting-based representation (a 30-graft system) to decouple scale management from the modulus chain, so that the scheme runs on efficient 32-bit GPU integer arithmetic while retaining flexible rescaling. It also optimizes ciphertext-ciphertext matrix multiplication for the Q.K^T attention product by removing redundant linear transformations and merging overlapping rotation patterns. End-to-end encrypted BERT inference on RTE takes 107.94 s on an A100, 5.77x faster than Thor and 3.18x faster than Powerformer, and the whole pipeline fits in a 24 GB RTX 4090, where it runs about 1.65x faster than on the A100.
Abstract¶
Privacy-preserving machine learning (PPML) enables inference over sensitive data without exposing raw inputs, with CKKS being a widely adopted scheme for approximate arithmetic. However, existing CKKS implementations are primarily based on 64-bit residue number system (RNS) representations, creating a mismatch with modern GPUs optimized for 32-bit integer arithmetic. This mismatch introduces substantial computational overhead, limiting the practicality of encrypted transformer inference. In this work, we present Zephyr, a GPU-efficient framework for homomorphic transformer inference via 32-bit arithmetic and grafting. Zephyr revisits the design of CKKS under GPU constraints and introduces a grafting-based representation that decouples scale management from the modulus chain. By constructing the RNS basis entirely with 30-bit primes and managing scale through auxiliary graft structures, Zephyr enables flexible rescaling while remaining compatible with efficient 32-bit GPU execution. Compared to Cheddar (Choi et al., ASPLOS’26), a representative GPU-oriented CKKS design based on fixed 25-30 prime systems, our approach simplifies modulus management and enables more flexible operations across different levels, while reducing rescaling overhead at the cost of additional convolution overhead. We further optimize ciphertext-ciphertext matrix multiplication (CCMM), a major bottleneck in encrypted transformer inference, by eliminating redundant linear transformations and merging overlapping rotation patterns in attention computation. Our theoretical and empirical analysis demonstrates that grafting-based 32-bit CKKS provides a practical and flexible design point for GPU-accelerated PPML inference.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | grafting-based 32-bit CKKS design (30-graft system) whose RNS basis consists only of 30-bit primes, decoupling scale management from the modulus chain | llm | 0.85 |
| contribution | optimized ciphertext-ciphertext matrix multiplication for attention (Q.K^T) that eliminates redundant linear transformations and merges overlapping rotation patterns | llm | 0.85 |
| contribution | BS-GS configurations that minimize switching keys under a 24 GB GPU memory budget | llm | 0.85 |
| contribution | theoretical and empirical comparison with Cheddar, Thor, Powerformer and Bolt on end-to-end BERT inference | llm | 0.85 |
| schemes | CKKS | llm | 0.85 |
| models | BERT | llm | 0.85 |
| datasets | RTE | llm | 0.85 |
| tasks | private transformer inference | llm | 0.85 |
| tasks | text entailment classification (RTE) | llm | 0.85 |
| hardware | NVIDIA A100 80GB GPU | llm | 0.85 |
| hardware | NVIDIA RTX 4090 24GB GPU | llm | 0.85 |
| threat model | outsourced cloud inference on encrypted inputs | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| code url | https://github.com/tuneinsight/lattigo (URL found in the full text) | regex | 0.50 |
| limitations | memory-bound operations benefit less from reduced arithmetic precision, so the observed 3.18x is below the theoretical 4.76x | llm | 0.85 |
| limitations | grafting reduces rescaling overhead at the cost of additional convolution overhead | llm | 0.85 |
| limitations | minimizing switching keys increases the number of key-switching operations for CPMM | llm | 0.85 |
| limitations | speed-up comparisons are influenced by implementation differences across systems | llm | 0.85 |
| llm relevance | high | llm | 0.85 |
| key result | end-to-end inference latency: 107.94 s (BERT, RTE benchmark, NVIDIA A100 80GB) | llm | 0.85 |
| key result | speed-up over Thor: 5.77x (623.39 s -> 107.94 s) (end-to-end BERT, RTE, A100) | llm | 0.85 |
| key result | speed-up over Powerformer: 3.18x (344.32 s -> 107.94 s) (end-to-end BERT, RTE, A100) | llm | 0.85 |
| key result | bootstrapping time: 36.54 s (vs 337.86 s Thor, 103.72 s Powerformer) (end-to-end BERT, A100) | llm | 0.85 |
| key result | RTX 4090 vs A100 speed-up: about 1.65x (end-to-end inference within 24 GB GPU memory) | llm | 0.85 |
| key result | expected speed-up of 30-graft system: about 4.76x (theoretical) (under an identical-implementation assumption vs 64-bit RNS) | llm | 0.85 |
System card: Zephyr¶
| Group | 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK) |
| Family | CKKS with 32-bit RNS + grafting |
| Arithmetic | Approximate real (CKKS) |
| Interaction | Non-interactive FHE |
| Model scale | Transformer (see paper) |
| Status | Active SOTA |
| Relation to the SoK | Redesigns CKKS for 32-bit GPU integer arithmetic. |
| Note | Combines 32-bit words with grafting to remove the 64-bit RNS mismatch on GPUs. |
| Systems page | Zephyr |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 38 [p.1], MPC 4 [p.1]
- library: Cheddar 6 [p.1], Lattigo 3 [p.3]
- model: Transformer 7 [p.1], BERT 3 [p.8]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 1 [p.8]
- hardware: GPU 39 [p.1], NVIDIA A100 13 [p.8], NVIDIA RTX 10 [p.8], Multi-GPU 1 [p.11]
- technique: RNS 47 [p.1], Bootstrapping 19 [p.3], Rescaling 15 [p.1], Matrix multiplication 14 [p.1], Key switching 13 [p.2], NTT 13 [p.4], Attention 11 [p.3], Convolution 11 [p.1], LayerNorm 7 [p.3], Rotations 7 [p.1], Softmax 4 [p.3], Non-interactive 4 [p.1], GELU 3 [p.3]
- topic: Homomorphic encryption 29 [p.1], Large language models 6 [p.6], Private / secure inference 5 [p.1], Privacy-preserving machine learning 4 [p.1], Benchmark 3 [p.8], Open source 2 [p.10]
Related papers¶
- this paper follow up of Grafting: Decoupled Scale Factors and Modulus in RNS-CKKS — follow-up listed for scheme "Grafting (decoupled scale and modulus)"
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/1-124_zephyr-gpu-efficient-homomorphic-encryption-for-privacy-pres.pdf |
catalogue | 18 | 391.4 KB | d6c7bc04e127 |
|
references/schemes/s07-1_eprint-2026-932.pdf |
schemes | 18 | 391.4 KB | d6c7bc04e127 |
byte-identical to references/catalogue/1-124_zephyr-gpu-efficient-homomorphic-encryption-for-privacy-pres.pdf |
Database key: zephyr-gpu-efficient-homomorphic-encryption-for-privacy-preserving · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set zephyr-gpu-efficient-homomorphic-encryption-for-privacy-preserving COLUMN VALUE