CipherGPT: Secure Two-Party GPT Inference¶
Authors: Hou, Liu, Li et al.
Year: 2023
Venue: IACR ePrint 2023/1147
type: system collection: catalogue status: auto
Links: ePrint 2023/1147
Summary¶
Auto-extracted summary (unreviewed)
CipherGPT is presented as the first framework for secure two-party inference of GPT-style generative models, built from an sVOLE-based secure matrix multiplication that amortizes preprocessing across repeated decoding steps, a spline-based secure GELU protocol, and the first protocol for secure top-k sampling. Against Iron, Bolt and BumbleBee it reports up to 6.2x faster and 4.1x less bandwidth for matrix multiplication, and 1.8x faster, 2.5x less communication and 7.4x lower maximal ULP error for GELU, together with a full per-operation benchmark.
Abstract¶
ChatGPT is recognized as a significant revolution in the field of artificial intelligence, but it raises serious concerns regarding user privacy, as the data submitted by users may contain sensitive information. Existing solutions for secure inference face significant challenges in supporting GPT-like models due to the enormous number of model parameters and complex activation functions. In this paper, we develop CipherGPT, the first framework for secure two-party GPT inference, building upon a series of innovative protocols. First, we propose a secure matrix multiplication that is customized for GPT inference, achieving upto 6.2× speedup and 4.1× bandwidth reduction over SOTA. We also propose a novel protocol for securely computing GELU, surpassing SOTA by 1.8× in runtime, 2.5× in communication and 7.4× in precision. Furthermore, we come up with the first protocol for secure top-k sampling. We provide a full-fledged implementation and comprehensive benchmark for CipherGPT. In particular, we measure the runtime and communication for each individual operation, along with their corresponding proportions. We believe this can serve as a reference for future research in this area.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | sVOLE-based secure matrix multiplication customized for repeated GPT decoding | llm | 0.85 |
| contribution | secure GELU protocol using a 64-piece spline approximation | llm | 0.85 |
| contribution | first protocol for secure top-k sampling | llm | 0.85 |
| contribution | full implementation and per-operation runtime/communication benchmark | llm | 0.85 |
| schemes | additive HE (SEAL) | llm | 0.85 |
| schemes | secret sharing | llm | 0.85 |
| schemes | oblivious transfer | llm | 0.85 |
| schemes | sVOLE | llm | 0.85 |
| schemes | BOLE | llm | 0.85 |
| libraries | SEAL | llm | 0.85 |
| libraries | HEXL | llm | 0.85 |
| libraries | SPU (BumbleBee BOLE code) | llm | 0.85 |
| libraries | SIRNN (EzPC) | llm | 0.85 |
| models | GPT-2 | llm | 0.85 |
| tasks | private LLM inference | llm | 0.85 |
| tasks | text generation | llm | 0.85 |
| security level | 128-bit | llm | 0.85 |
| threat model | two-party client-server; server learns nothing about input, client learns nothing about model beyond result | llm | 0.85 |
| interaction | interactive (HE+MPC) | llm | 0.85 |
| code url | https://github.com/Microsoft/SEAL (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/intel/hexl (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/secretflow/spu (URL found in the full text) | regex | 0.50 |
| limitations | matrix-multiplication comparison amortizes preprocessing over many iterations, which the authors acknowledge may be considered unfair to baselines | llm | 0.85 |
| llm relevance | high | llm | 0.85 |
| key result | GELU runtime / communication: 30.56 s, 764.96 MB (2^20 elements of 37 bits) | llm | 0.85 |
| key result | GELU improvement over Bolt: 1.8x faster, 2.5x less communication, 7.4x lower max ULP error (5 vs 37) (64-piece spline) | llm | 0.85 |
| key result | amortized MatMul cost: 1462 ms, 8.2 MB (Z^{256x768} x Z^{768x768} over 37-bit ring, t=256 iterations) | llm | 0.85 |
| key result | MatMul improvement over BumbleBee: 6.2x runtime, 4.1x communication (t=1024 iterations) | llm | 0.85 |
| key result | MatMul improvement over Bolt: 6.4x runtime, 3.0x communication (t=256 iterations) | llm | 0.85 |
System card: CipherGPT¶
| Group | 5. Interactive MPC and HE+MPC hybrids |
| Family | VOLE + HE 2PC |
| Arithmetic | Fixed point |
| Interaction | Interactive 2PC |
| Model scale | GPT-2 |
| Status | Active |
| Relation to the SoK | First 2PC GPT generation framework. |
| Note | Model-preserving but heavy communication, about 25 GB per token. |
| Systems page | CipherGPT |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: Secret sharing 28 [p.1], MPC 9 [p.2], Garbled circuits 3 [p.1], BFV 2 [p.11], Zero-knowledge proofs 2 [p.14]
- library: EzPC / CrypTFlow 4 [p.3], Microsoft SEAL 3 [p.11]
- model: GPT 44 [p.1], Transformer 1 [p.8], BERT 1 [p.1], VGG 1 [p.13], RNN / LSTM 1 [p.15], SVM 1 [p.13], Linear regression 1 [p.13], Decision tree 1 [p.15], Random forest 1 [p.15]
- dataset: CIFAR-10 1 [p.13], CIFAR-100 1 [p.13], WikiText 1 [p.12]
- hardware: GPU 5 [p.13], FPGA 1 [p.13], Intel Xeon 1 [p.11]
- technique: GELU 43 [p.1], Lookup table 19 [p.2], LayerNorm 18 [p.8], Two-party computation 15 [p.1], Softmax 11 [p.8], Attention 11 [p.3], SIMD packing 11 [p.4], Matrix multiplication 7 [p.1], Rotations 6 [p.4], Sigmoid 3 [p.2], Three-party computation 3 [p.13], Semi-honest model 2 [p.3]
- assumption: RLWE 15 [p.4]
- topic: Homomorphic encryption 17 [p.1], Private / secure inference 10 [p.1], Benchmark 9 [p.1], Open source 7 [p.11], Large language models 6 [p.1]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/5-069_ciphergpt-secure-two-party-gpt-inference.pdf |
catalogue | 16 | 1.2 MB | a72323995279 |
Database key: ciphergpt-secure-two-party-gpt-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set ciphergpt-secure-two-party-gpt-inference COLUMN VALUE