STIP: Efficient and secure non-interactive transformer inference via compact packing¶
Authors: Z. Wang; R. Chen; X. Gao; Y. Wang; L. Liu; Z. Lan; Z. Wang; S. Fu; Q. Wang; X. Huang
Year: 2026
Venue: IACR ePrint 2026/174
type: system collection: catalogue, sok-references status: auto
Links: ePrint 2026/174
Summary¶
Auto-extracted summary (unreviewed)
STIP is a non-interactive RNS-CKKS framework for secure transformer inference that introduces three packing strategies: real-imaginary hybrid packing (halving rotations in ciphertext-ciphertext matmuls), dual-head packing (doubling attention projection throughput) and adaptive multi-column packing (filling slots to cut ciphertext count), and replaces Softmax by a Gaussian kernel with an inverse-free LayerNorm to avoid max and inverse computations. Built on NEXUS/Euston with Phantom (GPU) and SEAL (CPU), it achieves about 1.6x end-to-end speed-up over Euston on BERT-base, LLaMA-3-8B and GPT-2-1.5B on an A100 while keeping GLUE accuracy within a fraction of a percent of plaintext.
Abstract¶
Secure TransFormer Inference (STFI) for LLMs aims to protect both user inputs and model parameters. Fully Homomorphic Encryption (FHE) offers a promising approach for STFI due to its noninteractivity, which eliminates communication overhead. However, FHE-based STFI incurs significant computational costs compared to plaintext inference. Recent advancements have accelerated inference by optimizing packing strategies and reducing the number of rotations. Despite these improvements, several challenges persist, including excessive rotations in ciphertext-ciphertext matrix multiplications (CCMMs), low input/output projection throughput, and expensive maximum/inverse operations, as well as wasted storage slots and inflated ciphertext counts due to sparse packing. To address these issues, we propose STIP, an efficient and secure non-interactive transformer inference framework that incorporates three novel packing strategies: (1) Real-Imaginary Hybrid Packing (RIHP) halves the rotation costs of CCMMs by enabling the simultaneous computation of two output results within the real and imaginary components; (2) Dual-Head Packing (DHP) maps adjacent heads to the real and imaginary components, doubling the throughput of attention projections; and (3) Adaptive Multi-Column Packing (AMCP) packs multiple heads into a single ciphertext, maximizing slot occupancy to reduce the total ciphertext count and thereby enhance computational parallelism. Moreover, for non-linear layers, we employ the Gaussian Kernel instead of Softmax, eliminating the need for maximum value searches and inverse operations, supported by a column-packed RIHP-based L2-norm algorithm. We reformulate LayerNorm into an inverse-free form by exploiting scale-invariance. Experimental results on a GPU show that STIP achieves approximately 1.6× speedup over the SOTA scheme Euston (S&P ’26) on BERT-base, LLAMA-3-8B, and GPT-2-1.5B.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Real-Imaginary Hybrid Packing (RIHP) halving rotation cost of ciphertext-ciphertext matrix multiplication | llm | 0.90 |
| contribution | Dual-Head Packing (DHP) doubling throughput of attention projections | llm | 0.90 |
| contribution | Adaptive Multi-Column Packing (AMCP) maximizing slot occupancy and reducing ciphertext count | llm | 0.90 |
| contribution | Gaussian-kernel attention replacing Softmax with a column-packed RIHP-based L2-norm | llm | 0.90 |
| contribution | inverse-free LayerNorm via scale invariance | llm | 0.90 |
| contribution | GPU and CPU implementation evaluated on BERT-base, LLaMA-3-8B and GPT-2-1.5B | llm | 0.90 |
| schemes | CKKS | llm | 0.90 |
| libraries | Phantom | llm | 0.90 |
| libraries | SEAL | llm | 0.90 |
| models | BERT-base | llm | 0.90 |
| models | LLaMA-3-8B | llm | 0.90 |
| models | GPT-2-1.5B | llm | 0.90 |
| datasets | GLUE | llm | 0.90 |
| datasets | RTE | llm | 0.90 |
| datasets | SST-2 | llm | 0.90 |
| datasets | QNLI | llm | 0.90 |
| tasks | private transformer inference | llm | 0.90 |
| tasks | text classification | llm | 0.90 |
| hardware | NVIDIA A100 80GB GPU | llm | 0.90 |
| hardware | AMD EPYC 7542 CPU (32 threads) | llm | 0.90 |
| security level | 128-bit | llm | 0.90 |
| threat model | two-party client-server; protects client input from the server and model parameters from the client | llm | 0.90 |
| interaction | non-interactive FHE | llm | 0.90 |
| code url | https://github.com/zju-abclab/NEXUS (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/FLL-Lab/Euston (URL found in the full text) | regex | 0.50 |
| llm relevance | high | llm | 0.90 |
| key result | end-to-end speed-up over Euston: 1.6x / 1.57x / 1.58x (BERT-base / LLaMA-3-8B / GPT-2-1.5B on A100 GPU, N=2^16, 1763-bit modulus) | llm | 0.90 |
| key result | CCMM (QK^T) time reduction: 45.1% (GPU), 33.1% (CPU) vs MOAI; >85% (GPU), 78% (CPU) vs Euston/NEXUS (BERT-base micro-benchmark) | llm | 0.90 |
| key result | ciphertext storage for Q,K,V: 576 MB vs 2,304 MB for Euston (75% reduction) (12 heads, m=128, t=64) | llm | 0.90 |
| key result | accuracy SST-2: 92.62% (STIP) vs 92.73% plaintext GK+ReLU (BERT) | llm | 0.90 |
| key result | accuracy SST-2: 94.01% (STIP) vs 94.39% plaintext GK+ReLU (LLaMA) | llm | 0.90 |
| key result | non-linear activation latency: 3.84 ms constant at small batch sizes (AMCP, GPU) | llm | 0.90 |
System card: STIP¶
| Group | 0. CKKS LLM frameworks surveyed in the SoK |
| Family | CKKS · Model-modifying (N-MOD) |
| Arithmetic | Approximate real (CKKS) |
| Interaction | non-interactive FHE |
| Model scale | BERT-Base, GPT-2-1.5B, Llama-3-8B |
| Library | SEAL + Phantom |
| Hardware | A100 GPU |
| Packing | L-CYC (real + imaginary slots) |
| Latency | 64 s (BERT-Base) / 142 s (GPT-2-1.5B) / 407 s (Llama-3-8B) amortized |
| Open source | partial |
| Status | Active SOTA |
| Relation to the SoK | Uses both real and imaginary slots; Gaussian-kernel attention; delayed LayerNorm. |
| Note | Compact packing; model modifications required. |
| Systems page | STIP |
SoK reference [20]¶
Z. Wang, R. Chen, X. Gao, Y. Wang, L. Liu, Z. Lan, Z. Wang, S. Fu, Q. Wang, and X. Huang, “Stip: Efficient and secure noninteractive transformer inference via compact packing,” Cryptology ePrint Archive, 2026.
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 9 [p.2], MPC 3 [p.1], Secret sharing 2 [p.1]
- library: Phantom 2 [p.10], Microsoft SEAL 1 [p.10], Concrete (Zama) 1 [p.17]
- model: BERT 20 [p.1], GPT 12 [p.1], Transformer 4 [p.1], LLaMA 2 [p.1]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 8 [p.10], SST-2 4 [p.10], GLUE 3 [p.10]
- hardware: GPU 43 [p.1], NVIDIA A100 1 [p.10], AMD EPYC 1 [p.10], CUDA 1 [p.10], Multi-GPU 1 [p.13]
- technique: Rotations 66 [p.1], LayerNorm 35 [p.1], Attention 28 [p.1], Softmax 26 [p.1], Bootstrapping 20 [p.3], ReLU 18 [p.3], Key switching 10 [p.2], GELU 9 [p.3], Non-interactive 8 [p.1], RNS 4 [p.2], Matrix multiplication 4 [p.1], SIMD packing 3 [p.3], Rescaling 2 [p.3], Knowledge distillation 2 [p.12], Two-party computation 2 [p.1]
- task: Medical / healthcare 2 [p.1], Finance 2 [p.1]
- topic: Benchmark 12 [p.2], Homomorphic encryption 8 [p.1], Large language models 5 [p.1], Private / secure inference 2 [p.1], Open source 2 [p.10]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/0-016_stip-efficient-and-secure-non-interactive-transformer-infere.pdf |
catalogue | 17 | 2.2 MB | 6b988ed2afbc |
|
references/sok-references/020_stip-efficient-and-secure-non-interactive-transformer-infere.pdf |
sok-references | 17 | 2.2 MB | 6b988ed2afbc |
byte-identical to references/catalogue/0-016_stip-efficient-and-secure-non-interactive-transformer-infere.pdf |
Database key: stip-efficient-and-secure-non-interactive-transformer-inference-via · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set stip-efficient-and-secure-non-interactive-transformer-inference-via COLUMN VALUE