Skip to content

Nimbus: Secure and Efficient Two-Party Inference for Transformers

Authors: Zhengyi Li; Kang Yang; Jin Tan; Wen-jie Lu; Haoqi Wu; Xiao Wang; Yu Yu; Derun Zhao; Yancheng Zheng; Minyi Guo; Jingwen Leng
Year: 2024
Venue: NeurIPS 2024

type: system collection: catalogue status: auto

Links: arXiv:2411.15707

Summary

Auto-extracted summary (unreviewed)

Nimbus is a two-party inference framework for Transformers combining homomorphic encryption and additive secret sharing. For linear layers it introduces a client-side outer-product (COP) protocol with row-wise encoding, in which the server sends encrypted weights once at setup so the online phase needs no input communication and produces compact output ciphertexts (2.9-12.5x faster than the SOTA protocol); for non-linear layers it uses distribution-aware low-degree polynomial approximations of GELU and Softmax (2.9-4.0x faster than SOTA approximations with 0.08% average accuracy loss). End-to-end BERT-base inference is 2.7-4.7x faster than the SOTA two-party system across network settings.

Abstract

Transformer models have gained significant attention due to their power in machine learning tasks. Their extensive deployment has raised concerns about the potential leakage of sensitive information during inference. However, when being applied to Transformers, existing approaches based on secure two-party computation (2PC) bring about efficiency limitations in two folds: (1) resource-intensive matrix multiplications in linear layers, and (2) complex non-linear activation functions like $\mathsf{GELU}$ and $\mathsf{Softmax}$. This work presents a new two-party inference framework $\mathsf{Nimbus}$ for Transformer models. For the linear layer, we propose a new 2PC paradigm along with an encoding approach to securely compute matrix multiplications based on an outer-product insight, which achieves $2.9\times \sim 12.5\times$ performance improvements compared to the state-of-the-art (SOTA) protocol. For the non-linear layer, through a new observation of utilizing the input distribution, we propose an approach of low-degree polynomial approximation for $\mathsf{GELU}$ and $\mathsf{Softmax}$, which improves the performance of the SOTA polynomial approximation by $2.9\times \sim 4.0\times$, where the average accuracy loss of our approach is 0.08\% compared to the non-2PC inference without privacy. Compared with the SOTA two-party inference, $\mathsf{Nimbus}$ improves the end-to-end performance of \bert{} inference by $2.7\times \sim 4.7\times$ across different network settings.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution client-side outer product (COP) protocol with row-wise encoding for secure matrix multiplication llm 0.85
contribution encrypted weights transferred once at setup, eliminating online input communication and sparse window-encoding overhead llm 0.85
contribution distribution-aware low-degree polynomial approximation of GELU and Softmax with fewer rounds and less communication llm 0.85
contribution 2.7-4.7x end-to-end speedup of BERT-base two-party inference over the state of the art llm 0.85
contribution implementation on SecretFlow SPU llm 0.85
schemes homomorphic encryption llm 0.85
schemes additive secret sharing llm 0.85
libraries SecretFlow SPU llm 0.85
models BERT-base llm 0.85
tasks secure transformer inference llm 0.85
tasks language understanding llm 0.85
threat model two-party client-server; client learns only the inference result, server learns nothing about the client's input llm 0.85
interaction interactive (HE+MPC) llm 0.85
code url https://github.com/secretflow/spu.2 (URL found in the full text) regex 0.50
code url https://github.com/secretflow/spu llm 0.85
llm relevance high llm 0.85
key result linear layer speedup: 2.9-12.5x (secure matrix multiplication vs SOTA 2PC protocol) llm 0.85
key result non-linear layer speedup: 2.9-4.0x (GELU and Softmax vs SOTA polynomial approximation) llm 0.85
key result accuracy loss: 0.08% average (compared with non-private inference) llm 0.85
key result end-to-end speedup: 2.7-4.7x (BERT-base inference vs SOTA two-party inference across network settings) llm 0.85

System card: Nimbus

Group 5. Interactive MPC and HE+MPC hybrids
Family HE + 2PC, outer-product encoding
Arithmetic Fixed point
Interaction Interactive 2PC
Model scale BERT-Base
Status Active SOTA
Relation to the SoK Client-side outer-product HE encoding for linear layers.
Note Uses low-degree piecewise polynomials for non-linear layers.
Code https://github.com/secretflow/spu
Systems page Nimbus

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Secret sharing 17 [p.1], MPC 17 [p.3], Oblivious transfer 2 [p.3]
  • library: SecretFlow-SPU 7 [p.8], HuggingFace 1 [p.8]
  • model: Transformer 40 [p.1], LLaMA 2 [p.13], BERT 1 [p.14], GPT 1 [p.14], CNN 1 [p.10], RNN / LSTM 1 [p.12]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 14 [p.8], GLUE 5 [p.8], SST-2 2 [p.8]
  • hardware: GPU 1 [p.27]
  • technique: GELU 53 [p.1], Softmax 40 [p.1], Two-party computation 29 [p.1], Matrix multiplication 24 [p.1], NTT 23 [p.4], Polynomial approximation 9 [p.1], Fine-tuning 7 [p.8], LayerNorm 4 [p.3], Quantization 4 [p.11], Taylor expansion 3 [p.6], Semi-honest model 3 [p.2], ReLU 2 [p.23], Attention 2 [p.14], Convolution 2 [p.1], Knowledge distillation 2 [p.10]
  • assumption: RLWE 11 [p.13], Lattice-based 4 [p.3]
  • topic: Homomorphic encryption 39 [p.1], Open source 8 [p.2], Private / secure inference 6 [p.7], Large language models 5 [p.11], Benchmark 4 [p.8]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/5-073_nimbus-secure-and-efficient-two-party-inference-for-transfor.pdf catalogue 29 1.3 MB 8f13e3a0e869

Database key: nimbus-secure-and-efficient-two-party-inference-for-transformers · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set nimbus-secure-and-efficient-two-party-inference-for-transformers COLUMN VALUE