EncFormer: Secure and Efficient Transformer Inference over Encrypted Data¶
Authors: Yufan Zhu; Chao Jin; Khin Mi Mi Aung; Xiaokui Xiao
Year: 2026
Venue: arXiv 2604.09975
type: system collection: catalogue status: auto
Links: arXiv:2604.09975
Summary¶
Auto-extracted summary (unreviewed)
EncFormer is a two-party hybrid FHE-MPC framework for private Transformer inference that introduces Stage Compatible Patterns so CKKS kernels compose without repacking, a cost model around a minimal-conversion baseline for placing FHE-MPC boundaries, a secure CKKS-MPC conversion exploiting complex slots, and communication-efficient MPC protocols for nonlinearities, with GPU optimizations. On GPT- and BERT-style models it reports 1.4x-30.4x lower online MPC communication and 1.3x-9.8x lower end-to-end latency than prior hybrid systems, and 1.9x-3.5x lower latency than FHE-only pipelines on BERT-base, with near-plaintext accuracy on selected GLUE tasks.
Abstract¶
Transformer inference in machine-learning-as-a-service (MLaaS) raises privacy concerns for sensitive user inputs. Prior secure solutions that combine fully homomorphic encryption (FHE) and secure multiparty computation (MPC) are bottlenecked by inefficient FHE kernels, communication-heavy MPC protocols, and expensive FHE-MPC conversions. We present EncFormer, a two-party private Transformer inference framework that introduces Stage Compatible Patterns so that FHE kernels compose efficiently, reducing repacking and conversions. EncFormer also provides a cost analysis model built around a minimal-conversion baseline, enabling principled selection of FHE-MPC boundaries. To further reduce communication, EncFormer proposes a secure complex CKKS-MPC conversion protocol and designs communication-efficient MPC protocols for nonlinearities. With GPU optimizations, evaluations on GPT- and BERT-style models show that EncFormer achieves 1.4x-30.4x lower online MPC communication and 1.3x-9.8x lower end-to-end latency against prior hybrid FHE-MPC systems, and 1.9x-3.5x lower end-to-end latency on BERT-base than FHE-only pipelines under a matched backend, while maintaining near-plaintext accuracy on selected GLUE tasks.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Stage Compatible Patterns for composing FHE kernels with fewer repackings and conversions | llm | 0.70 |
| contribution | Cost analysis model for principled FHE-MPC boundary selection | llm | 0.70 |
| contribution | Secure complex-slot CKKS-to-MPC conversion protocol | llm | 0.70 |
| contribution | Communication-efficient MPC protocols for Transformer nonlinearities | llm | 0.70 |
| contribution | GPU-optimized hybrid pipeline evaluated on GPT- and BERT-style models | llm | 0.70 |
| schemes | CKKS | llm | 0.70 |
| schemes | secret sharing | llm | 0.70 |
| models | BERT-base | llm | 0.70 |
| models | GPT-style models | llm | 0.70 |
| datasets | GLUE (selected tasks) | llm | 0.70 |
| tasks | private transformer inference | llm | 0.70 |
| hardware | GPU (unspecified in excerpt) | llm | 0.70 |
| threat model | two-party client / untrusted model provider | llm | 0.70 |
| interaction | interactive (HE+MPC) | llm | 0.70 |
| code url | https://github.com/Desilo/liberate-fhe (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/Microsoft/SEAL (URL found in the full text) | regex | 0.50 |
| llm relevance | high | llm | 0.70 |
| key result | online MPC communication reduction: 1.4x-30.4x (vs prior hybrid FHE-MPC systems) | llm | 0.70 |
| key result | end-to-end latency reduction: 1.3x-9.8x (vs prior hybrid FHE-MPC systems, GPT- and BERT-style models) | llm | 0.70 |
| key result | end-to-end latency reduction: 1.9x-3.5x (vs FHE-only pipelines on BERT-base, matched backend) | llm | 0.70 |
System card: EncFormer¶
| Group | 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK) |
| Family | CKKS + MPC hybrid |
| Arithmetic | Approximate + secret sharing |
| Interaction | Interactive 2PC |
| Model scale | GPT and BERT style models |
| Status | Active SOTA |
| Relation to the SoK | Hybrid that optimizes FHE-to-MPC boundaries with complex CKKS conversion. |
| Note | Cost model chooses where to switch between FHE and MPC to minimize conversions. |
| Systems page | EncFormer |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: MPC 184 [p.1], CKKS 112 [p.1], Secret sharing 5 [p.2]
- library: Liberate.FHE 16 [p.2], EzPC / CrypTFlow 5 [p.10], Microsoft SEAL 2 [p.14], ABY / ABY3 2 [p.13], CrypTen 2 [p.10], Concrete (Zama) 1 [p.4], Phantom 1 [p.14], FIDESlib 1 [p.15], EVA / CHET 1 [p.13], MP-SPDZ 1 [p.2], TensorFlow 1 [p.15], HuggingFace 1 [p.15]
- model: BERT 55 [p.1], Transformer 31 [p.1], GPT 20 [p.1], CNN 5 [p.13], MLP 2 [p.22], LLaMA 1 [p.15], RNN / LSTM 1 [p.15]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 8 [p.2], GLUE 7 [p.1], SST-2 3 [p.2]
- hardware: GPU 12 [p.1], NVIDIA A100 6 [p.7], CUDA 1 [p.14]
- technique: GELU 49 [p.2], Rotations 42 [p.2], Softmax 31 [p.2], Two-party computation 13 [p.1], LayerNorm 11 [p.2], Bootstrapping 10 [p.1], Key switching 9 [p.5], RNS 9 [p.3], Rescaling 5 [p.3], Non-interactive 5 [p.12], Semi-honest model 5 [p.2], ReLU 4 [p.22], Matrix multiplication 4 [p.5], Fine-tuning 4 [p.10], SIMD packing 3 [p.2], Relinearization 2 [p.2], Attention 2 [p.4]
- task: Language modeling / generation 9 [p.2], Medical / healthcare 6 [p.1], Finance 2 [p.1]
- topic: Homomorphic encryption 154 [p.1], Private / secure inference 13 [p.1], Benchmark 3 [p.11], Open source 3 [p.14]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/1-129_encformer-secure-and-efficient-transformer-inference-over-en.pdf |
catalogue | 22 | 3.0 MB | 96621b250fb3 |
Database key: encformer-secure-and-efficient-transformer-inference-over-encrypted-data · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set encformer-secure-and-efficient-transformer-inference-over-encrypted-data COLUMN VALUE