Skip to content

CipherGPT: Secure Two-Party GPT Inference

Authors: Hou, Liu, Li et al.
Year: 2023
Venue: IACR ePrint 2023/1147

type: system collection: catalogue status: auto

Links: ePrint 2023/1147

Summary

Auto-extracted summary (unreviewed)

CipherGPT is presented as the first framework for secure two-party inference of GPT-style generative models, built from an sVOLE-based secure matrix multiplication that amortizes preprocessing across repeated decoding steps, a spline-based secure GELU protocol, and the first protocol for secure top-k sampling. Against Iron, Bolt and BumbleBee it reports up to 6.2x faster and 4.1x less bandwidth for matrix multiplication, and 1.8x faster, 2.5x less communication and 7.4x lower maximal ULP error for GELU, together with a full per-operation benchmark.

Abstract

ChatGPT is recognized as a significant revolution in the field of artificial intelligence, but it raises serious concerns regarding user privacy, as the data submitted by users may contain sensitive information. Existing solutions for secure inference face significant challenges in supporting GPT-like models due to the enormous number of model parameters and complex activation functions. In this paper, we develop CipherGPT, the first framework for secure two-party GPT inference, building upon a series of innovative protocols. First, we propose a secure matrix multiplication that is customized for GPT inference, achieving upto 6.2× speedup and 4.1× bandwidth reduction over SOTA. We also propose a novel protocol for securely computing GELU, surpassing SOTA by 1.8× in runtime, 2.5× in communication and 7.4× in precision. Furthermore, we come up with the first protocol for secure top-k sampling. We provide a full-fledged implementation and comprehensive benchmark for CipherGPT. In particular, we measure the runtime and communication for each individual operation, along with their corresponding proportions. We believe this can serve as a reference for future research in this area.
Source: pdf

Extracted details

Field Value Source Confidence
contribution sVOLE-based secure matrix multiplication customized for repeated GPT decoding llm 0.85
contribution secure GELU protocol using a 64-piece spline approximation llm 0.85
contribution first protocol for secure top-k sampling llm 0.85
contribution full implementation and per-operation runtime/communication benchmark llm 0.85
schemes additive HE (SEAL) llm 0.85
schemes secret sharing llm 0.85
schemes oblivious transfer llm 0.85
schemes sVOLE llm 0.85
schemes BOLE llm 0.85
libraries SEAL llm 0.85
libraries HEXL llm 0.85
libraries SPU (BumbleBee BOLE code) llm 0.85
libraries SIRNN (EzPC) llm 0.85
models GPT-2 llm 0.85
tasks private LLM inference llm 0.85
tasks text generation llm 0.85
security level 128-bit llm 0.85
threat model two-party client-server; server learns nothing about input, client learns nothing about model beyond result llm 0.85
interaction interactive (HE+MPC) llm 0.85
code url https://github.com/Microsoft/SEAL (URL found in the full text) regex 0.50
code url https://github.com/intel/hexl (URL found in the full text) regex 0.50
code url https://github.com/secretflow/spu (URL found in the full text) regex 0.50
limitations matrix-multiplication comparison amortizes preprocessing over many iterations, which the authors acknowledge may be considered unfair to baselines llm 0.85
llm relevance high llm 0.85
key result GELU runtime / communication: 30.56 s, 764.96 MB (2^20 elements of 37 bits) llm 0.85
key result GELU improvement over Bolt: 1.8x faster, 2.5x less communication, 7.4x lower max ULP error (5 vs 37) (64-piece spline) llm 0.85
key result amortized MatMul cost: 1462 ms, 8.2 MB (Z^{256x768} x Z^{768x768} over 37-bit ring, t=256 iterations) llm 0.85
key result MatMul improvement over BumbleBee: 6.2x runtime, 4.1x communication (t=1024 iterations) llm 0.85
key result MatMul improvement over Bolt: 6.4x runtime, 3.0x communication (t=256 iterations) llm 0.85

System card: CipherGPT

Group 5. Interactive MPC and HE+MPC hybrids
Family VOLE + HE 2PC
Arithmetic Fixed point
Interaction Interactive 2PC
Model scale GPT-2
Status Active
Relation to the SoK First 2PC GPT generation framework.
Note Model-preserving but heavy communication, about 25 GB per token.
Systems page CipherGPT

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Secret sharing 28 [p.1], MPC 9 [p.2], Garbled circuits 3 [p.1], BFV 2 [p.11], Zero-knowledge proofs 2 [p.14]
  • library: EzPC / CrypTFlow 4 [p.3], Microsoft SEAL 3 [p.11]
  • model: GPT 44 [p.1], Transformer 1 [p.8], BERT 1 [p.1], VGG 1 [p.13], RNN / LSTM 1 [p.15], SVM 1 [p.13], Linear regression 1 [p.13], Decision tree 1 [p.15], Random forest 1 [p.15]
  • dataset: CIFAR-10 1 [p.13], CIFAR-100 1 [p.13], WikiText 1 [p.12]
  • hardware: GPU 5 [p.13], FPGA 1 [p.13], Intel Xeon 1 [p.11]
  • technique: GELU 43 [p.1], Lookup table 19 [p.2], LayerNorm 18 [p.8], Two-party computation 15 [p.1], Softmax 11 [p.8], Attention 11 [p.3], SIMD packing 11 [p.4], Matrix multiplication 7 [p.1], Rotations 6 [p.4], Sigmoid 3 [p.2], Three-party computation 3 [p.13], Semi-honest model 2 [p.3]
  • assumption: RLWE 15 [p.4]
  • topic: Homomorphic encryption 17 [p.1], Private / secure inference 10 [p.1], Benchmark 9 [p.1], Open source 7 [p.11], Large language models 6 [p.1]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/5-069_ciphergpt-secure-two-party-gpt-inference.pdf catalogue 16 1.2 MB a72323995279

Database key: ciphergpt-secure-two-party-gpt-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set ciphergpt-secure-two-party-gpt-inference COLUMN VALUE