Skip to content

Centaur: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference

Authors: Jinglong Luo; Guanzhong Chen; Yehong Zhang; Shiyu Liu; Hui Wang; Yue Yu; Xun Zhou; Yuan Qi; Zenglin Xu
Year: 2025
Venue: ACL Findings 2025

type: system collection: catalogue status: auto

Links: arXiv:2412.10652

Summary

Auto-extracted summary (unreviewed)

Centaur is a privacy-preserving transformer inference framework that protects model parameters with random permutations and inference data with secret sharing, so linear layers become communication-free plaintext-share multiplications and nonlinear layers (softmax, GeLU, LayerNorm) are evaluated in plaintext on permuted values. The authors report plaintext-level accuracy, 5.0-30.4x faster inference than SMPC-based approaches, and strong empirical resistance to data reconstruction attacks on BERT-large and GPT-2-large, while acknowledging that permutation-based privacy is not provably secure.

Abstract

With the growing deployment of pre-trained models like Transformers on cloud platforms, privacy concerns about model parameters and inference data are intensifying. Existing Privacy-Preserving Transformer Inference (PPTI) frameworks face the "impossible trinity" of balancing privacy, efficiency, and performance: Secure Multi-Party Computation (SMPC)-based approaches ensure strong privacy but suffer from high computational overhead and performance losses; Conversely, permutation-based methods achieve near-plaintext efficiency and accuracy but compromise privacy by exposing sensitive model parameters and intermediate results. Bridging this gap with a single approach presents substantial challenges, motivating the introduction of CENTAUR, a groundbreaking PPTI framework that seamlessly integrates random permutations and SMPC to address the "impossible trinity". By designing efficient PPTI algorithms tailored to the structural properties of Transformer models, CENTAUR achieves an unprecedented balance among privacy, efficiency, and performance. Our experiments demonstrate CENTAUR's ability to resist diverse data reconstruction attacks, achieve plaintext-level inference accuracy, and boost inference speed by 5.0-30.4 times, unlocking new possibilities for secure and efficient AI deployment.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution hybrid framework combining random permutation of parameters with secret sharing of data llm 0.85
contribution communication-free linear layers via plaintext-share multiplication with permuted weights llm 0.85
contribution permuted-state evaluation of softmax, GeLU and LayerNorm with two communication rounds llm 0.85
contribution privacy-preserving permutation protocol to re-permute intermediate shares llm 0.85
contribution empirical evaluation against data reconstruction attacks (SIP, EIA, BRE) llm 0.85
schemes secret sharing llm 0.85
schemes random permutation llm 0.85
models BERT-large llm 0.85
models GPT-2-large llm 0.85
datasets QNLI llm 0.85
datasets WikiText-103 llm 0.85
tasks private transformer inference llm 0.85
tasks NLP llm 0.85
threat model two-party (model developer and cloud platform); empirical rather than provable security llm 0.85
interaction other llm 0.85
limitations random permutation cannot directly achieve theoretical (provable) security llm 0.85
limitations security is supported empirically through attack experiments rather than formal proofs llm 0.85
llm relevance high llm 0.85
key result inference speed-up: 5.0x to 30.4x (vs SMPC-based PPTI) llm 0.85
key result accuracy: plaintext-level (transformer inference) llm 0.85
key result SIP attack success (avg over intermediate results): 84.09 without protection vs 3.86 with Centaur (BERT-large on QNLI) llm 0.85
key result EIA attack success (avg): 90.99 without protection vs 6.39 with Centaur (GPT-2-large on WikiText-103) llm 0.85

System card: Centaur

Group 5. Interactive MPC and HE+MPC hybrids
Family Random permutation + 2PC
Arithmetic Fixed point
Interaction Interactive 2PC
Model scale BERT, GPT-2
Status Active
Relation to the SoK Permutes weights so non-linear steps run in plaintext.
Note Weaker privacy than full cryptography but much faster.
Systems page Centaur

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Secret sharing 17 [p.2], MPC 5 [p.2]
  • library: CrypTen 3 [p.3]
  • model: Transformer 42 [p.1], GPT 35 [p.3], BERT 14 [p.6], LLaMA 13 [p.7], ViT 2 [p.11], CNN 2 [p.15], RNN / LSTM 1 [p.17]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 28 [p.6], GLUE 2 [p.6], WikiText 1 [p.17]
  • hardware: NVIDIA A100 2 [p.6], GPU 2 [p.6], Intel Xeon 1 [p.18]
  • technique: Softmax 17 [p.5], Matrix multiplication 17 [p.3], GELU 15 [p.5], LayerNorm 11 [p.6], Two-party computation 6 [p.2], Semi-honest model 6 [p.4], Fine-tuning 5 [p.2], Attention 3 [p.2], KV cache 3 [p.9], Knowledge distillation 2 [p.9], Three-party computation 2 [p.4]
  • task: Clustering 3 [p.11]
  • topic: Private / secure inference 14 [p.3], Large language models 5 [p.1], Homomorphic encryption 2 [p.9], Benchmark 2 [p.6]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/5-077_centaur-bridging-the-impossible-trinity-of-privacy-efficienc.pdf catalogue 20 1.4 MB 5a0dcfc6257e

Database key: centaur-bridging-the-impossible-trinity-of-privacy-efficiency-and · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set centaur-bridging-the-impossible-trinity-of-privacy-efficiency-and COLUMN VALUE