Skip to content

STIP: Efficient and secure non-interactive transformer inference via compact packing

Authors: Z. Wang; R. Chen; X. Gao; Y. Wang; L. Liu; Z. Lan; Z. Wang; S. Fu; Q. Wang; X. Huang
Year: 2026
Venue: IACR ePrint 2026/174

type: system collection: catalogue, sok-references status: auto

Links: ePrint 2026/174

Summary

Auto-extracted summary (unreviewed)

STIP is a non-interactive RNS-CKKS framework for secure transformer inference that introduces three packing strategies: real-imaginary hybrid packing (halving rotations in ciphertext-ciphertext matmuls), dual-head packing (doubling attention projection throughput) and adaptive multi-column packing (filling slots to cut ciphertext count), and replaces Softmax by a Gaussian kernel with an inverse-free LayerNorm to avoid max and inverse computations. Built on NEXUS/Euston with Phantom (GPU) and SEAL (CPU), it achieves about 1.6x end-to-end speed-up over Euston on BERT-base, LLaMA-3-8B and GPT-2-1.5B on an A100 while keeping GLUE accuracy within a fraction of a percent of plaintext.

Abstract

Secure TransFormer Inference (STFI) for LLMs aims to protect both user inputs and model parameters. Fully Homomorphic Encryption (FHE) offers a promising approach for STFI due to its noninteractivity, which eliminates communication overhead. However, FHE-based STFI incurs significant computational costs compared to plaintext inference. Recent advancements have accelerated inference by optimizing packing strategies and reducing the number of rotations. Despite these improvements, several challenges persist, including excessive rotations in ciphertext-ciphertext matrix multiplications (CCMMs), low input/output projection throughput, and expensive maximum/inverse operations, as well as wasted storage slots and inflated ciphertext counts due to sparse packing. To address these issues, we propose STIP, an efficient and secure non-interactive transformer inference framework that incorporates three novel packing strategies: (1) Real-Imaginary Hybrid Packing (RIHP) halves the rotation costs of CCMMs by enabling the simultaneous computation of two output results within the real and imaginary components; (2) Dual-Head Packing (DHP) maps adjacent heads to the real and imaginary components, doubling the throughput of attention projections; and (3) Adaptive Multi-Column Packing (AMCP) packs multiple heads into a single ciphertext, maximizing slot occupancy to reduce the total ciphertext count and thereby enhance computational parallelism. Moreover, for non-linear layers, we employ the Gaussian Kernel instead of Softmax, eliminating the need for maximum value searches and inverse operations, supported by a column-packed RIHP-based L2-norm algorithm. We reformulate LayerNorm into an inverse-free form by exploiting scale-invariance. Experimental results on a GPU show that STIP achieves approximately 1.6× speedup over the SOTA scheme Euston (S&P ’26) on BERT-base, LLAMA-3-8B, and GPT-2-1.5B.
Source: pdf

Extracted details

Field Value Source Confidence
contribution Real-Imaginary Hybrid Packing (RIHP) halving rotation cost of ciphertext-ciphertext matrix multiplication llm 0.90
contribution Dual-Head Packing (DHP) doubling throughput of attention projections llm 0.90
contribution Adaptive Multi-Column Packing (AMCP) maximizing slot occupancy and reducing ciphertext count llm 0.90
contribution Gaussian-kernel attention replacing Softmax with a column-packed RIHP-based L2-norm llm 0.90
contribution inverse-free LayerNorm via scale invariance llm 0.90
contribution GPU and CPU implementation evaluated on BERT-base, LLaMA-3-8B and GPT-2-1.5B llm 0.90
schemes CKKS llm 0.90
libraries Phantom llm 0.90
libraries SEAL llm 0.90
models BERT-base llm 0.90
models LLaMA-3-8B llm 0.90
models GPT-2-1.5B llm 0.90
datasets GLUE llm 0.90
datasets RTE llm 0.90
datasets SST-2 llm 0.90
datasets QNLI llm 0.90
tasks private transformer inference llm 0.90
tasks text classification llm 0.90
hardware NVIDIA A100 80GB GPU llm 0.90
hardware AMD EPYC 7542 CPU (32 threads) llm 0.90
security level 128-bit llm 0.90
threat model two-party client-server; protects client input from the server and model parameters from the client llm 0.90
interaction non-interactive FHE llm 0.90
code url https://github.com/zju-abclab/NEXUS (URL found in the full text) regex 0.50
code url https://github.com/FLL-Lab/Euston (URL found in the full text) regex 0.50
llm relevance high llm 0.90
key result end-to-end speed-up over Euston: 1.6x / 1.57x / 1.58x (BERT-base / LLaMA-3-8B / GPT-2-1.5B on A100 GPU, N=2^16, 1763-bit modulus) llm 0.90
key result CCMM (QK^T) time reduction: 45.1% (GPU), 33.1% (CPU) vs MOAI; >85% (GPU), 78% (CPU) vs Euston/NEXUS (BERT-base micro-benchmark) llm 0.90
key result ciphertext storage for Q,K,V: 576 MB vs 2,304 MB for Euston (75% reduction) (12 heads, m=128, t=64) llm 0.90
key result accuracy SST-2: 92.62% (STIP) vs 92.73% plaintext GK+ReLU (BERT) llm 0.90
key result accuracy SST-2: 94.01% (STIP) vs 94.39% plaintext GK+ReLU (LLaMA) llm 0.90
key result non-linear activation latency: 3.84 ms constant at small batch sizes (AMCP, GPU) llm 0.90

System card: STIP

Group 0. CKKS LLM frameworks surveyed in the SoK
Family CKKS · Model-modifying (N-MOD)
Arithmetic Approximate real (CKKS)
Interaction non-interactive FHE
Model scale BERT-Base, GPT-2-1.5B, Llama-3-8B
Library SEAL + Phantom
Hardware A100 GPU
Packing L-CYC (real + imaginary slots)
Latency 64 s (BERT-Base) / 142 s (GPT-2-1.5B) / 407 s (Llama-3-8B) amortized
Open source partial
Status Active SOTA
Relation to the SoK Uses both real and imaginary slots; Gaussian-kernel attention; delayed LayerNorm.
Note Compact packing; model modifications required.
Systems page STIP

SoK reference [20]

Z. Wang, R. Chen, X. Gao, Y. Wang, L. Liu, Z. Lan, Z. Wang, S. Fu, Q. Wang, and X. Huang, “Stip: Efficient and secure noninteractive transformer inference via compact packing,” Cryptology ePrint Archive, 2026.

Listed on the reference page of the site.

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 9 [p.2], MPC 3 [p.1], Secret sharing 2 [p.1]
  • library: Phantom 2 [p.10], Microsoft SEAL 1 [p.10], Concrete (Zama) 1 [p.17]
  • model: BERT 20 [p.1], GPT 12 [p.1], Transformer 4 [p.1], LLaMA 2 [p.1]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 8 [p.10], SST-2 4 [p.10], GLUE 3 [p.10]
  • hardware: GPU 43 [p.1], NVIDIA A100 1 [p.10], AMD EPYC 1 [p.10], CUDA 1 [p.10], Multi-GPU 1 [p.13]
  • technique: Rotations 66 [p.1], LayerNorm 35 [p.1], Attention 28 [p.1], Softmax 26 [p.1], Bootstrapping 20 [p.3], ReLU 18 [p.3], Key switching 10 [p.2], GELU 9 [p.3], Non-interactive 8 [p.1], RNS 4 [p.2], Matrix multiplication 4 [p.1], SIMD packing 3 [p.3], Rescaling 2 [p.3], Knowledge distillation 2 [p.12], Two-party computation 2 [p.1]
  • task: Medical / healthcare 2 [p.1], Finance 2 [p.1]
  • topic: Benchmark 12 [p.2], Homomorphic encryption 8 [p.1], Large language models 5 [p.1], Private / secure inference 2 [p.1], Open source 2 [p.10]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/0-016_stip-efficient-and-secure-non-interactive-transformer-infere.pdf catalogue 17 2.2 MB 6b988ed2afbc
references/sok-references/020_stip-efficient-and-secure-non-interactive-transformer-infere.pdf sok-references 17 2.2 MB 6b988ed2afbc byte-identical to references/catalogue/0-016_stip-efficient-and-secure-non-interactive-transformer-infere.pdf

Database key: stip-efficient-and-secure-non-interactive-transformer-inference-via · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set stip-efficient-and-secure-non-interactive-transformer-inference-via COLUMN VALUE