Skip to content

Iron: Private Inference on Transformers

Authors: Hao, Li, Chen, Xing, Xu, Zhang
Year: 2022
Venue: NeurIPS 2022

type: system collection: catalogue status: auto

Links: link

Summary

Auto-extracted summary (unreviewed)

Iron is a two-party cryptographic framework for private inference on Transformer models where the client holds the input and the server holds the proprietary weights. It introduces a homomorphic-encryption matrix multiplication protocol with a compact polynomial packing that cuts communication by a factor of sqrt(m) over Cheetah, plus secret-sharing-based protocols for Softmax, GELU and LayerNorm that roughly halve communication and computation while staying numerically precise. End to end, Iron needs 3-14x less communication and 3-11x less runtime than prior art with no loss of plaintext accuracy.

Abstract

We initiate the study of private inference on Transformer-based models in the client-server setting, where clients have private inputs and servers hold proprietary models. Our main contribution is to provide several new secure protocols for matrix multiplication and complex non-linear functions like Softmax, GELU activations, and LayerNorm, which are critical components of Transformers. Specifically, we first propose a customized homomorphic encryption-based protocol for matrix multiplication that crucially relies on a novel compact packing technique. This design achieves √m× less communication (m is the number of rows of the output matrix) over the most efficient work. Second, we design efficient protocols for three non-linear functions via integrating advanced underlying protocols and specialized optimizations. Compared to the state-of-the-art protocols, our recipes reduce about half of the communication and computation overhead. Furthermore, all protocols are numerically precise, which preserve the model accuracy of plaintext. These techniques together allow us to implement Iron, an efficient Transformer-based private inference framework. Experiments conducted on several real-world datasets and models demonstrate that Iron achieves 3 ∼14× less communication and 3 ∼11× less runtime compared to the prior art.
Source: pdf

Extracted details

Field Value Source Confidence
contribution HE-based secure matrix multiplication protocol with compact input packing (sqrt(m) less communication than Cheetah) llm 0.80
contribution Communication-efficient 2PC protocols for Softmax, GELU and LayerNorm llm 0.80
contribution Numerically precise protocols that preserve plaintext model accuracy llm 0.80
contribution Iron end-to-end private Transformer inference framework llm 0.80
schemes additively homomorphic encryption (RLWE-based) llm 0.80
schemes secret sharing llm 0.80
libraries SEAL llm 0.80
libraries EMP-toolkit llm 0.80
libraries EzPC llm 0.80
models BERT llm 0.80
tasks private Transformer inference llm 0.80
tasks NLP llm 0.80
threat model two-party client-server; client has private inputs, server holds proprietary model llm 0.80
interaction interactive (HE+MPC) llm 0.80
code url https://github.com/microsoft/SEAL (URL found in the full text) regex 0.50
code url https://github.com/mpc-msri/EzPC (URL found in the full text) regex 0.50
code url https://github.com/google-research/bert (URL found in the full text) regex 0.50
llm relevance high llm 0.80
key result communication reduction: 3-14x less (end-to-end vs prior art, several datasets and models) llm 0.80
key result runtime reduction: 3-11x less (end-to-end vs prior art) llm 0.80
key result matrix multiplication communication: sqrt(m)x less (m = rows of output matrix) (vs Cheetah; factor m when mnk <= N) llm 0.80
key result non-linear protocol overhead: about half the communication and computation (Softmax, GELU, LayerNorm vs state-of-the-art protocols) llm 0.80

System card: Iron

Group 5. Interactive MPC and HE+MPC hybrids
Family HE (BFV) + OT/SS 2PC
Arithmetic Exact fixed point
Interaction Interactive 2PC
Model scale BERT-Base
Status Foundational
Relation to the SoK First efficient 2PC Transformer with HE matmul.
Note Model-preserving but gigabytes of communication per inference.
Systems page Iron

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Secret sharing 8 [p.2], MPC 7 [p.4], Oblivious transfer 4 [p.3], BFV 2 [p.3], Differential privacy 2 [p.3]
  • library: MP-SPDZ 10 [p.2], Microsoft SEAL 2 [p.8], EzPC / CrypTFlow 2 [p.8], TensorFlow 2 [p.8]
  • model: Transformer 35 [p.1], BERT 23 [p.1], CNN 5 [p.2], ViT 2 [p.1], RNN / LSTM 2 [p.2], Mixture of experts 2 [p.10], GPT 1 [p.1]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 8 [p.8], SST-2 2 [p.8], GLUE 2 [p.2]
  • hardware: Intel Xeon 1 [p.8], GPU 1 [p.14]
  • technique: Matrix multiplication 37 [p.2], LayerNorm 31 [p.1], Softmax 29 [p.1], GELU 24 [p.1], Attention 6 [p.1], ReLU 5 [p.2], Sigmoid 4 [p.7], Quantization 3 [p.10], Two-party computation 3 [p.2], Convolution 2 [p.1], Pruning 2 [p.10]
  • task: Question answering 2 [p.1]
  • topic: Private / secure inference 20 [p.1], Homomorphic encryption 7 [p.1], Open source 4 [p.12], Benchmark 3 [p.2]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/5-066_iron-private-inference-on-transformers.pdf catalogue 14 723.3 KB 3628e177bd62

Database key: iron-private-inference-on-transformers · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set iron-private-inference-on-transformers COLUMN VALUE