Skip to content

MPC-Minimized Secure LLM Inference

Authors: Deevashwer Rathee; Dacheng Li; Ion Stoica; Hao Zhang; Raluca Popa
Year: 2024
Venue: arXiv 2024

type: system collection: catalogue status: auto

Links: arXiv:2408.03561

Summary

Auto-extracted summary (unreviewed)

Marill is a fine-tuning framework that changes the architecture of an LLM during fine-tuning (layer freezing so that public pre-trained layers run outside MPC, LoRA to cut multiplications, head merging to cut non-arithmetic attention operations, combined with distillation) so that far fewer expensive operations must be executed inside MPC at secure-inference time. Evaluated on OpenLLaMA-3B-v2 and DeepSeek-Coder-1.3B across 2PC, 3PC and 2PC-Dealer frameworks, Marill-generated models run 3.6-11.3x faster and use 2.4-6.9x less communication than standard fine-tuned models while typically retaining over 90% of downstream performance on MTBench, HumanEval and WMT22.

Abstract

Many inference services based on large language models (LLMs) pose a privacy concern, either revealing user prompts to the service or the proprietary weights to the user. Secure inference offers a solution to this problem through secure multi-party computation (MPC), however, it is still impractical for modern LLM workload due to the large overhead imposed by MPC. To address this overhead, we propose Marill, a framework that adapts LLM fine-tuning to minimize MPC usage during secure inference. Marill introduces high-level architectural changes during fine-tuning that significantly reduce the number of expensive operations needed within MPC during inference, by removing some and relocating others outside MPC without compromising security. As a result, Marill-generated models are more efficient across all secure inference protocols and our approach complements MPC-friendly approximations for such operations. Compared to standard fine-tuning, Marill results in 3.6-11.3x better runtime and 2.4-6.9x better communication during secure inference across various MPC settings, while typically preserving over 90% performance across downstream tasks.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution Marill: MPC-minimizing fine-tuning framework for secure LLM inference llm 0.85
contribution layer freezing to keep public pre-trained layers (and embedding) outside MPC llm 0.85
contribution LoRA fine-tuning to reduce integer multiplications inside MPC llm 0.85
contribution head merging to reduce non-arithmetic operations in self-attention llm 0.85
contribution evaluation across 2PC, 3PC and 2PC-Dealer secure inference frameworks llm 0.85
contribution shown to be complementary to MPC-friendly approximations llm 0.85
schemes MPC (2PC, 3PC, 2PC-Dealer settings) llm 0.85
models OpenLLaMA-3B-v2 llm 0.85
models DeepSeek-Coder-1.3B-base llm 0.85
datasets ShareGPT llm 0.85
datasets MTBench llm 0.85
datasets MagiCoder llm 0.85
datasets HumanEval llm 0.85
datasets ParroT llm 0.85
datasets WMT22 (De-En) llm 0.85
tasks chatbot llm 0.85
tasks code generation llm 0.85
tasks machine translation llm 0.85
tasks secure LLM inference llm 0.85
hardware Intel Xeon Platinum 8173M (16 vCPUs, 128 GB RAM) llm 0.85
hardware NVIDIA V100 GPU (16 GB) llm 0.85
threat model user and inference service provider; two-party, three-party and two-party-with-dealer MPC settings; prompts and fine-tuned weights kept private llm 0.85
interaction pure MPC llm 0.85
code url https://huggingface.co/blog/encrypted-llm (URL found in the full text) regex 0.50
limitations 2PC experiments restricted to small sequence lengths (64) because of memory requirements llm 0.85
limitations head merging gives little improvement in 2PC at small sequence lengths llm 0.85
limitations communication improvement from LoRA in the 2PC-Dealer setting is an implementation artefact llm 0.85
limitations accuracy trade-off: models retain typically over 90%, not all, of the fine-tuned performance llm 0.85
llm relevance high llm 0.85
key result secure inference runtime improvement: 3.6-11.3x (versus standard fine-tuning across all MPC settings and inference stages) llm 0.85
key result communication improvement: 2.4-6.9x (versus standard fine-tuning across all MPC settings and inference stages) llm 0.85
key result downstream performance retained: typically over 90% (MTBench, HumanEval, WMT22) llm 0.85
key result pre-filling runtime improvement from LoRA: 4.9x (2PC, OpenLLaMA-3B-v2 (92% of 2PC runtime in multiplications)) llm 0.85
key result head-merging improvement: 2.1-2.4x runtime, 2.6-2.8x communication (3PC and 2PC-Dealer, pre-filling stage) llm 0.85
key result MTBench score: 4.95 fully fine-tuned vs 4.77 (LF=0.5), 4.72 (LoRA=64), 4.3 (HM=4), 1.25 zero-shot (OpenLLaMA-3B-v2, Figure 5) llm 0.85

System card: MPC-minimized secure LLM inference (Marill)

Group 5. Interactive MPC and HE+MPC hybrids
Family MPC + LLM fine-tuning co-design
Arithmetic Fixed point
Interaction Interactive 2PC/3PC
Model scale Llama-2-7B
Status Active SOTA
Relation to the SoK Fine-tunes LLMs so fewer layers need MPC.
Note Model-modifying. Combines public pre-trained layers with private ones.
Systems page MPC-minimized secure LLM inference (Marill)

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: MPC 110 [p.1], Secret sharing 8 [p.4], Zero-knowledge proofs 2 [p.17]
  • library: ABY / ABY3 1 [p.13], CrypTen 1 [p.12], EzPC / CrypTFlow 1 [p.13]
  • model: LoRA 32 [p.3], Transformer 7 [p.1], LLaMA 6 [p.1], BERT 3 [p.12], GPT 2 [p.1], RNN / LSTM 1 [p.13]
  • hardware: GPU 6 [p.3], NVIDIA V100 2 [p.7], Intel Xeon 1 [p.7]
  • technique: Fine-tuning 66 [p.1], Two-party computation 49 [p.3], Pruning 21 [p.3], Three-party computation 20 [p.3], Attention 14 [p.3], ReLU 10 [p.4], Softmax 7 [p.3], Semi-honest model 7 [p.4], Malicious model 7 [p.4], GELU 3 [p.3], LayerNorm 2 [p.5], Matrix multiplication 2 [p.3]
  • task: Language modeling / generation 25 [p.1], Machine translation 6 [p.3]
  • topic: Private / secure inference 76 [p.1], Large language models 34 [p.1], Open source 15 [p.1], Benchmark 11 [p.3], Homomorphic encryption 2 [p.10]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/5-076_mpc-minimized-secure-llm-inference.pdf catalogue 20 1.2 MB 3f197aef58f6

Database key: mpc-minimized-secure-llm-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set mpc-minimized-secure-llm-inference COLUMN VALUE