Skip to content

SIGMA: Secure GPT Inference with Function Secret Sharing

Authors: Kanav Gupta; Neha Jawalkar; Ananta Mukherjee; Nishanth Chandran; Divya Gupta; Ashish Panwar; Rahul Sharma
Year: 2024
Venue: PETS 2024

type: system collection: catalogue status: auto

Links: DOI · ePrint 2023/1269 · Semantic Scholar

Summary

Auto-extracted summary (unreviewed)

Sigma is the first end-to-end system for secure two-party transformer inference built on function secret sharing (FSS), contributing new FSS protocols for Softmax, GeLU and SiLU and accelerating them on GPUs. It improves secure transformer inference latency by 12-19x over the prior state of the art that uses preprocessing and GPUs, and presents the first secure inference of GPT-style models, running Llama2-13B in 38 s and GPT-2 in 1.5 s (the metadata abstract reports 44 s and 1.6 s).

Abstract

Secure 2-party computation (2PC) enables secure inference that offers protection for both proprietary machine learning (ML) models and sensitive inputs to them. However, the existing secure inference solutions suffer from high latency and communication overheads, particularly for transformers. Function secret sharing (FSS) is a recent paradigm for obtaining efficient 2PC protocols with a preprocessing phase. We provide Sigma, the first end-to-end system for secure transformer inference based on FSS. By constructing new FSS-based protocols for complex machine learning functionalities, such as Softmax, GeLU and SiLU, and also accelerating their computation on GPUs, Sigma improves the latency of secure inference of transformers by 11 - 19x over the state-of-the-art that uses preprocessing and GPUs. We present the first secure inference of generative pre-trained transformer (GPT) models. In particular, Sigma executes Meta's Llama2 (available on HuggingFace) with 13 billion parameters in 44 seconds and GPT2 in 1.6 seconds
Source: crossref

Extracted details

Field Value Source Confidence
contribution First end-to-end FSS-based system for secure transformer inference llm 0.80
contribution New FSS protocols for complex ML functionalities: Softmax, GeLU and SiLU llm 0.80
contribution GPU acceleration of FSS-based secure computation llm 0.80
contribution First secure inference of generative pre-trained transformers (GPT-2, Llama2-13B) llm 0.80
schemes function secret sharing llm 0.80
schemes secret sharing llm 0.80
libraries EzPC (GPU-MPC) llm 0.80
libraries CUTLASS llm 0.80
libraries cryptoTools llm 0.80
models GPT-2 llm 0.80
models LLaMA-2-13B llm 0.80
tasks private transformer/LLM inference llm 0.80
tasks GPT inference llm 0.80
hardware GPU llm 0.80
threat model two-party (2PC) client-server with a preprocessing phase; protects both model and client input llm 0.80
interaction pure MPC llm 0.80
code url https://github.com/mpc-msri/EzPC/tree/master/GPU-MPC/experiments/sigma (URL found in the full text) regex 0.50
code url https://github.com/NVIDIA/cutlass (URL found in the full text) regex 0.50
code url https://huggingface.co/gpt2 (URL found in the full text) regex 0.50
code url https://github.com/mpc-msri/EzPC/tree/master/GPU-MPC/experiments/sigma llm 0.80
limitations Grotto could not be compared experimentally because its source code is unavailable llm 0.80
llm relevance high llm 0.80
key result latency improvement: 12-19x (metadata abstract: 11-19x) (vs state-of-the-art secure transformer inference using preprocessing and GPUs) llm 0.80
key result latency: 38 s (metadata abstract: 44 s) (secure inference of Llama2-13B) llm 0.80
key result latency: 1.5 s (metadata abstract: 1.6 s) (secure inference of GPT-2) llm 0.80

System card: SIGMA

Group 5. Interactive MPC and HE+MPC hybrids
Family Function secret sharing on GPU
Arithmetic Fixed point
Interaction Interactive 2PC with dealer
Model scale Llama-2-13B
Status Active SOTA
Relation to the SoK GPU-accelerated FSS with a trusted dealer for preprocessing.
Note Seconds to tens of seconds per token for 13B models. Requires offline correlated randomness.
Code https://github.com/mpc-msri/EzPC
Systems page SIGMA

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Function secret sharing 55 [p.1], Secret sharing 28 [p.1], MPC 21 [p.1]
  • library: CrypTen 54 [p.1], PyTorch 9 [p.8], EzPC / CrypTFlow 4 [p.2], HuggingFace 3 [p.1], MP-SPDZ 1 [p.14], ABY / ABY3 1 [p.15], TensorFlow 1 [p.14]
  • model: GPT 52 [p.1], BERT 42 [p.1], Transformer 14 [p.1], CNN 6 [p.2], LLaMA 5 [p.14], VGG 1 [p.15], RNN / LSTM 1 [p.15], Logistic regression 1 [p.14]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 8 [p.11], SST-2 4 [p.11], GLUE 2 [p.11], ImageNet 1 [p.15], LAMBADA 1 [p.15]
  • hardware: GPU 97 [p.1], NVIDIA A100 1 [p.14], CUDA 1 [p.10]
  • technique: GELU 60 [p.1], Softmax 41 [p.1], Lookup table 41 [p.2], LayerNorm 32 [p.4], Two-party computation 27 [p.1], ReLU 17 [p.2], Matrix multiplication 8 [p.4], Non-interactive 6 [p.5], Sigmoid 5 [p.4], Attention 4 [p.4], Semi-honest model 4 [p.3], Malicious model 4 [p.14], Model extraction / stealing 4 [p.15], Fine-tuning 2 [p.1]
  • topic: Private / secure inference 53 [p.1], Benchmark 6 [p.7], Open source 4 [p.2], Privacy-preserving machine learning 2 [p.15]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/5-070_sigma-secure-gpt-inference-with-function-secret-sharing.pdf catalogue 19 912.0 KB 3a2989f1da36

Database key: sigma-secure-gpt-inference-with-function-secret-sharing · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set sigma-secure-gpt-inference-with-function-secret-sharing COLUMN VALUE