Centaur: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference¶
Authors: Jinglong Luo; Guanzhong Chen; Yehong Zhang; Shiyu Liu; Hui Wang; Yue Yu; Xun Zhou; Yuan Qi; Zenglin Xu
Year: 2025
Venue: ACL Findings 2025
type: system collection: catalogue status: auto
Links: arXiv:2412.10652
Summary¶
Auto-extracted summary (unreviewed)
Centaur is a privacy-preserving transformer inference framework that protects model parameters with random permutations and inference data with secret sharing, so linear layers become communication-free plaintext-share multiplications and nonlinear layers (softmax, GeLU, LayerNorm) are evaluated in plaintext on permuted values. The authors report plaintext-level accuracy, 5.0-30.4x faster inference than SMPC-based approaches, and strong empirical resistance to data reconstruction attacks on BERT-large and GPT-2-large, while acknowledging that permutation-based privacy is not provably secure.
Abstract¶
With the growing deployment of pre-trained models like Transformers on cloud platforms, privacy concerns about model parameters and inference data are intensifying. Existing Privacy-Preserving Transformer Inference (PPTI) frameworks face the "impossible trinity" of balancing privacy, efficiency, and performance: Secure Multi-Party Computation (SMPC)-based approaches ensure strong privacy but suffer from high computational overhead and performance losses; Conversely, permutation-based methods achieve near-plaintext efficiency and accuracy but compromise privacy by exposing sensitive model parameters and intermediate results. Bridging this gap with a single approach presents substantial challenges, motivating the introduction of CENTAUR, a groundbreaking PPTI framework that seamlessly integrates random permutations and SMPC to address the "impossible trinity". By designing efficient PPTI algorithms tailored to the structural properties of Transformer models, CENTAUR achieves an unprecedented balance among privacy, efficiency, and performance. Our experiments demonstrate CENTAUR's ability to resist diverse data reconstruction attacks, achieve plaintext-level inference accuracy, and boost inference speed by 5.0-30.4 times, unlocking new possibilities for secure and efficient AI deployment.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | hybrid framework combining random permutation of parameters with secret sharing of data | llm | 0.85 |
| contribution | communication-free linear layers via plaintext-share multiplication with permuted weights | llm | 0.85 |
| contribution | permuted-state evaluation of softmax, GeLU and LayerNorm with two communication rounds | llm | 0.85 |
| contribution | privacy-preserving permutation protocol to re-permute intermediate shares | llm | 0.85 |
| contribution | empirical evaluation against data reconstruction attacks (SIP, EIA, BRE) | llm | 0.85 |
| schemes | secret sharing | llm | 0.85 |
| schemes | random permutation | llm | 0.85 |
| models | BERT-large | llm | 0.85 |
| models | GPT-2-large | llm | 0.85 |
| datasets | QNLI | llm | 0.85 |
| datasets | WikiText-103 | llm | 0.85 |
| tasks | private transformer inference | llm | 0.85 |
| tasks | NLP | llm | 0.85 |
| threat model | two-party (model developer and cloud platform); empirical rather than provable security | llm | 0.85 |
| interaction | other | llm | 0.85 |
| limitations | random permutation cannot directly achieve theoretical (provable) security | llm | 0.85 |
| limitations | security is supported empirically through attack experiments rather than formal proofs | llm | 0.85 |
| llm relevance | high | llm | 0.85 |
| key result | inference speed-up: 5.0x to 30.4x (vs SMPC-based PPTI) | llm | 0.85 |
| key result | accuracy: plaintext-level (transformer inference) | llm | 0.85 |
| key result | SIP attack success (avg over intermediate results): 84.09 without protection vs 3.86 with Centaur (BERT-large on QNLI) | llm | 0.85 |
| key result | EIA attack success (avg): 90.99 without protection vs 6.39 with Centaur (GPT-2-large on WikiText-103) | llm | 0.85 |
System card: Centaur¶
| Group | 5. Interactive MPC and HE+MPC hybrids |
| Family | Random permutation + 2PC |
| Arithmetic | Fixed point |
| Interaction | Interactive 2PC |
| Model scale | BERT, GPT-2 |
| Status | Active |
| Relation to the SoK | Permutes weights so non-linear steps run in plaintext. |
| Note | Weaker privacy than full cryptography but much faster. |
| Systems page | Centaur |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: Secret sharing 17 [p.2], MPC 5 [p.2]
- library: CrypTen 3 [p.3]
- model: Transformer 42 [p.1], GPT 35 [p.3], BERT 14 [p.6], LLaMA 13 [p.7], ViT 2 [p.11], CNN 2 [p.15], RNN / LSTM 1 [p.17]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 28 [p.6], GLUE 2 [p.6], WikiText 1 [p.17]
- hardware: NVIDIA A100 2 [p.6], GPU 2 [p.6], Intel Xeon 1 [p.18]
- technique: Softmax 17 [p.5], Matrix multiplication 17 [p.3], GELU 15 [p.5], LayerNorm 11 [p.6], Two-party computation 6 [p.2], Semi-honest model 6 [p.4], Fine-tuning 5 [p.2], Attention 3 [p.2], KV cache 3 [p.9], Knowledge distillation 2 [p.9], Three-party computation 2 [p.4]
- task: Clustering 3 [p.11]
- topic: Private / secure inference 14 [p.3], Large language models 5 [p.1], Homomorphic encryption 2 [p.9], Benchmark 2 [p.6]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/5-077_centaur-bridging-the-impossible-trinity-of-privacy-efficienc.pdf |
catalogue | 20 | 1.4 MB | 5a0dcfc6257e |
Database key: centaur-bridging-the-impossible-trinity-of-privacy-efficiency-and · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set centaur-bridging-the-impossible-trinity-of-privacy-efficiency-and COLUMN VALUE