Skip to content

EncFormer: Secure and Efficient Transformer Inference over Encrypted Data

Authors: Yufan Zhu; Chao Jin; Khin Mi Mi Aung; Xiaokui Xiao
Year: 2026
Venue: arXiv 2604.09975

type: system collection: catalogue status: auto

Links: arXiv:2604.09975

Summary

Auto-extracted summary (unreviewed)

EncFormer is a two-party hybrid FHE-MPC framework for private Transformer inference that introduces Stage Compatible Patterns so CKKS kernels compose without repacking, a cost model around a minimal-conversion baseline for placing FHE-MPC boundaries, a secure CKKS-MPC conversion exploiting complex slots, and communication-efficient MPC protocols for nonlinearities, with GPU optimizations. On GPT- and BERT-style models it reports 1.4x-30.4x lower online MPC communication and 1.3x-9.8x lower end-to-end latency than prior hybrid systems, and 1.9x-3.5x lower latency than FHE-only pipelines on BERT-base, with near-plaintext accuracy on selected GLUE tasks.

Abstract

Transformer inference in machine-learning-as-a-service (MLaaS) raises privacy concerns for sensitive user inputs. Prior secure solutions that combine fully homomorphic encryption (FHE) and secure multiparty computation (MPC) are bottlenecked by inefficient FHE kernels, communication-heavy MPC protocols, and expensive FHE-MPC conversions. We present EncFormer, a two-party private Transformer inference framework that introduces Stage Compatible Patterns so that FHE kernels compose efficiently, reducing repacking and conversions. EncFormer also provides a cost analysis model built around a minimal-conversion baseline, enabling principled selection of FHE-MPC boundaries. To further reduce communication, EncFormer proposes a secure complex CKKS-MPC conversion protocol and designs communication-efficient MPC protocols for nonlinearities. With GPU optimizations, evaluations on GPT- and BERT-style models show that EncFormer achieves 1.4x-30.4x lower online MPC communication and 1.3x-9.8x lower end-to-end latency against prior hybrid FHE-MPC systems, and 1.9x-3.5x lower end-to-end latency on BERT-base than FHE-only pipelines under a matched backend, while maintaining near-plaintext accuracy on selected GLUE tasks.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution Stage Compatible Patterns for composing FHE kernels with fewer repackings and conversions llm 0.70
contribution Cost analysis model for principled FHE-MPC boundary selection llm 0.70
contribution Secure complex-slot CKKS-to-MPC conversion protocol llm 0.70
contribution Communication-efficient MPC protocols for Transformer nonlinearities llm 0.70
contribution GPU-optimized hybrid pipeline evaluated on GPT- and BERT-style models llm 0.70
schemes CKKS llm 0.70
schemes secret sharing llm 0.70
models BERT-base llm 0.70
models GPT-style models llm 0.70
datasets GLUE (selected tasks) llm 0.70
tasks private transformer inference llm 0.70
hardware GPU (unspecified in excerpt) llm 0.70
threat model two-party client / untrusted model provider llm 0.70
interaction interactive (HE+MPC) llm 0.70
code url https://github.com/Desilo/liberate-fhe (URL found in the full text) regex 0.50
code url https://github.com/Microsoft/SEAL (URL found in the full text) regex 0.50
llm relevance high llm 0.70
key result online MPC communication reduction: 1.4x-30.4x (vs prior hybrid FHE-MPC systems) llm 0.70
key result end-to-end latency reduction: 1.3x-9.8x (vs prior hybrid FHE-MPC systems, GPT- and BERT-style models) llm 0.70
key result end-to-end latency reduction: 1.9x-3.5x (vs FHE-only pipelines on BERT-base, matched backend) llm 0.70

System card: EncFormer

Group 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK)
Family CKKS + MPC hybrid
Arithmetic Approximate + secret sharing
Interaction Interactive 2PC
Model scale GPT and BERT style models
Status Active SOTA
Relation to the SoK Hybrid that optimizes FHE-to-MPC boundaries with complex CKKS conversion.
Note Cost model chooses where to switch between FHE and MPC to minimize conversions.
Systems page EncFormer

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: MPC 184 [p.1], CKKS 112 [p.1], Secret sharing 5 [p.2]
  • library: Liberate.FHE 16 [p.2], EzPC / CrypTFlow 5 [p.10], Microsoft SEAL 2 [p.14], ABY / ABY3 2 [p.13], CrypTen 2 [p.10], Concrete (Zama) 1 [p.4], Phantom 1 [p.14], FIDESlib 1 [p.15], EVA / CHET 1 [p.13], MP-SPDZ 1 [p.2], TensorFlow 1 [p.15], HuggingFace 1 [p.15]
  • model: BERT 55 [p.1], Transformer 31 [p.1], GPT 20 [p.1], CNN 5 [p.13], MLP 2 [p.22], LLaMA 1 [p.15], RNN / LSTM 1 [p.15]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 8 [p.2], GLUE 7 [p.1], SST-2 3 [p.2]
  • hardware: GPU 12 [p.1], NVIDIA A100 6 [p.7], CUDA 1 [p.14]
  • technique: GELU 49 [p.2], Rotations 42 [p.2], Softmax 31 [p.2], Two-party computation 13 [p.1], LayerNorm 11 [p.2], Bootstrapping 10 [p.1], Key switching 9 [p.5], RNS 9 [p.3], Rescaling 5 [p.3], Non-interactive 5 [p.12], Semi-honest model 5 [p.2], ReLU 4 [p.22], Matrix multiplication 4 [p.5], Fine-tuning 4 [p.10], SIMD packing 3 [p.2], Relinearization 2 [p.2], Attention 2 [p.4]
  • task: Language modeling / generation 9 [p.2], Medical / healthcare 6 [p.1], Finance 2 [p.1]
  • topic: Homomorphic encryption 154 [p.1], Private / secure inference 13 [p.1], Benchmark 3 [p.11], Open source 3 [p.14]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/1-129_encformer-secure-and-efficient-transformer-inference-over-en.pdf catalogue 22 3.0 MB 96621b250fb3

Database key: encformer-secure-and-efficient-transformer-inference-over-encrypted-data · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set encformer-secure-and-efficient-transformer-inference-over-encrypted-data COLUMN VALUE