Iron: Private Inference on Transformers¶
Authors: Hao, Li, Chen, Xing, Xu, Zhang
Year: 2022
Venue: NeurIPS 2022
type: system collection: catalogue status: auto
Links: link
Summary¶
Auto-extracted summary (unreviewed)
Iron is a two-party cryptographic framework for private inference on Transformer models where the client holds the input and the server holds the proprietary weights. It introduces a homomorphic-encryption matrix multiplication protocol with a compact polynomial packing that cuts communication by a factor of sqrt(m) over Cheetah, plus secret-sharing-based protocols for Softmax, GELU and LayerNorm that roughly halve communication and computation while staying numerically precise. End to end, Iron needs 3-14x less communication and 3-11x less runtime than prior art with no loss of plaintext accuracy.
Abstract¶
We initiate the study of private inference on Transformer-based models in the client-server setting, where clients have private inputs and servers hold proprietary models. Our main contribution is to provide several new secure protocols for matrix multiplication and complex non-linear functions like Softmax, GELU activations, and LayerNorm, which are critical components of Transformers. Specifically, we first propose a customized homomorphic encryption-based protocol for matrix multiplication that crucially relies on a novel compact packing technique. This design achieves √m× less communication (m is the number of rows of the output matrix) over the most efficient work. Second, we design efficient protocols for three non-linear functions via integrating advanced underlying protocols and specialized optimizations. Compared to the state-of-the-art protocols, our recipes reduce about half of the communication and computation overhead. Furthermore, all protocols are numerically precise, which preserve the model accuracy of plaintext. These techniques together allow us to implement Iron, an efficient Transformer-based private inference framework. Experiments conducted on several real-world datasets and models demonstrate that Iron achieves 3 ∼14× less communication and 3 ∼11× less runtime compared to the prior art.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | HE-based secure matrix multiplication protocol with compact input packing (sqrt(m) less communication than Cheetah) | llm | 0.80 |
| contribution | Communication-efficient 2PC protocols for Softmax, GELU and LayerNorm | llm | 0.80 |
| contribution | Numerically precise protocols that preserve plaintext model accuracy | llm | 0.80 |
| contribution | Iron end-to-end private Transformer inference framework | llm | 0.80 |
| schemes | additively homomorphic encryption (RLWE-based) | llm | 0.80 |
| schemes | secret sharing | llm | 0.80 |
| libraries | SEAL | llm | 0.80 |
| libraries | EMP-toolkit | llm | 0.80 |
| libraries | EzPC | llm | 0.80 |
| models | BERT | llm | 0.80 |
| tasks | private Transformer inference | llm | 0.80 |
| tasks | NLP | llm | 0.80 |
| threat model | two-party client-server; client has private inputs, server holds proprietary model | llm | 0.80 |
| interaction | interactive (HE+MPC) | llm | 0.80 |
| code url | https://github.com/microsoft/SEAL (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/mpc-msri/EzPC (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/google-research/bert (URL found in the full text) | regex | 0.50 |
| llm relevance | high | llm | 0.80 |
| key result | communication reduction: 3-14x less (end-to-end vs prior art, several datasets and models) | llm | 0.80 |
| key result | runtime reduction: 3-11x less (end-to-end vs prior art) | llm | 0.80 |
| key result | matrix multiplication communication: sqrt(m)x less (m = rows of output matrix) (vs Cheetah; factor m when mnk <= N) | llm | 0.80 |
| key result | non-linear protocol overhead: about half the communication and computation (Softmax, GELU, LayerNorm vs state-of-the-art protocols) | llm | 0.80 |
System card: Iron¶
| Group | 5. Interactive MPC and HE+MPC hybrids |
| Family | HE (BFV) + OT/SS 2PC |
| Arithmetic | Exact fixed point |
| Interaction | Interactive 2PC |
| Model scale | BERT-Base |
| Status | Foundational |
| Relation to the SoK | First efficient 2PC Transformer with HE matmul. |
| Note | Model-preserving but gigabytes of communication per inference. |
| Systems page | Iron |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: Secret sharing 8 [p.2], MPC 7 [p.4], Oblivious transfer 4 [p.3], BFV 2 [p.3], Differential privacy 2 [p.3]
- library: MP-SPDZ 10 [p.2], Microsoft SEAL 2 [p.8], EzPC / CrypTFlow 2 [p.8], TensorFlow 2 [p.8]
- model: Transformer 35 [p.1], BERT 23 [p.1], CNN 5 [p.2], ViT 2 [p.1], RNN / LSTM 2 [p.2], Mixture of experts 2 [p.10], GPT 1 [p.1]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 8 [p.8], SST-2 2 [p.8], GLUE 2 [p.2]
- hardware: Intel Xeon 1 [p.8], GPU 1 [p.14]
- technique: Matrix multiplication 37 [p.2], LayerNorm 31 [p.1], Softmax 29 [p.1], GELU 24 [p.1], Attention 6 [p.1], ReLU 5 [p.2], Sigmoid 4 [p.7], Quantization 3 [p.10], Two-party computation 3 [p.2], Convolution 2 [p.1], Pruning 2 [p.10]
- task: Question answering 2 [p.1]
- topic: Private / secure inference 20 [p.1], Homomorphic encryption 7 [p.1], Open source 4 [p.12], Benchmark 3 [p.2]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/5-066_iron-private-inference-on-transformers.pdf |
catalogue | 14 | 723.3 KB | 3628e177bd62 |
Database key: iron-private-inference-on-transformers · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set iron-private-inference-on-transformers COLUMN VALUE