MPC-Minimized Secure LLM Inference¶
Authors: Deevashwer Rathee; Dacheng Li; Ion Stoica; Hao Zhang; Raluca Popa
Year: 2024
Venue: arXiv 2024
type: system collection: catalogue status: auto
Links: arXiv:2408.03561
Summary¶
Auto-extracted summary (unreviewed)
Marill is a fine-tuning framework that changes the architecture of an LLM during fine-tuning (layer freezing so that public pre-trained layers run outside MPC, LoRA to cut multiplications, head merging to cut non-arithmetic attention operations, combined with distillation) so that far fewer expensive operations must be executed inside MPC at secure-inference time. Evaluated on OpenLLaMA-3B-v2 and DeepSeek-Coder-1.3B across 2PC, 3PC and 2PC-Dealer frameworks, Marill-generated models run 3.6-11.3x faster and use 2.4-6.9x less communication than standard fine-tuned models while typically retaining over 90% of downstream performance on MTBench, HumanEval and WMT22.
Abstract¶
Many inference services based on large language models (LLMs) pose a privacy concern, either revealing user prompts to the service or the proprietary weights to the user. Secure inference offers a solution to this problem through secure multi-party computation (MPC), however, it is still impractical for modern LLM workload due to the large overhead imposed by MPC. To address this overhead, we propose Marill, a framework that adapts LLM fine-tuning to minimize MPC usage during secure inference. Marill introduces high-level architectural changes during fine-tuning that significantly reduce the number of expensive operations needed within MPC during inference, by removing some and relocating others outside MPC without compromising security. As a result, Marill-generated models are more efficient across all secure inference protocols and our approach complements MPC-friendly approximations for such operations. Compared to standard fine-tuning, Marill results in 3.6-11.3x better runtime and 2.4-6.9x better communication during secure inference across various MPC settings, while typically preserving over 90% performance across downstream tasks.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Marill: MPC-minimizing fine-tuning framework for secure LLM inference | llm | 0.85 |
| contribution | layer freezing to keep public pre-trained layers (and embedding) outside MPC | llm | 0.85 |
| contribution | LoRA fine-tuning to reduce integer multiplications inside MPC | llm | 0.85 |
| contribution | head merging to reduce non-arithmetic operations in self-attention | llm | 0.85 |
| contribution | evaluation across 2PC, 3PC and 2PC-Dealer secure inference frameworks | llm | 0.85 |
| contribution | shown to be complementary to MPC-friendly approximations | llm | 0.85 |
| schemes | MPC (2PC, 3PC, 2PC-Dealer settings) | llm | 0.85 |
| models | OpenLLaMA-3B-v2 | llm | 0.85 |
| models | DeepSeek-Coder-1.3B-base | llm | 0.85 |
| datasets | ShareGPT | llm | 0.85 |
| datasets | MTBench | llm | 0.85 |
| datasets | MagiCoder | llm | 0.85 |
| datasets | HumanEval | llm | 0.85 |
| datasets | ParroT | llm | 0.85 |
| datasets | WMT22 (De-En) | llm | 0.85 |
| tasks | chatbot | llm | 0.85 |
| tasks | code generation | llm | 0.85 |
| tasks | machine translation | llm | 0.85 |
| tasks | secure LLM inference | llm | 0.85 |
| hardware | Intel Xeon Platinum 8173M (16 vCPUs, 128 GB RAM) | llm | 0.85 |
| hardware | NVIDIA V100 GPU (16 GB) | llm | 0.85 |
| threat model | user and inference service provider; two-party, three-party and two-party-with-dealer MPC settings; prompts and fine-tuned weights kept private | llm | 0.85 |
| interaction | pure MPC | llm | 0.85 |
| code url | https://huggingface.co/blog/encrypted-llm (URL found in the full text) | regex | 0.50 |
| limitations | 2PC experiments restricted to small sequence lengths (64) because of memory requirements | llm | 0.85 |
| limitations | head merging gives little improvement in 2PC at small sequence lengths | llm | 0.85 |
| limitations | communication improvement from LoRA in the 2PC-Dealer setting is an implementation artefact | llm | 0.85 |
| limitations | accuracy trade-off: models retain typically over 90%, not all, of the fine-tuned performance | llm | 0.85 |
| llm relevance | high | llm | 0.85 |
| key result | secure inference runtime improvement: 3.6-11.3x (versus standard fine-tuning across all MPC settings and inference stages) | llm | 0.85 |
| key result | communication improvement: 2.4-6.9x (versus standard fine-tuning across all MPC settings and inference stages) | llm | 0.85 |
| key result | downstream performance retained: typically over 90% (MTBench, HumanEval, WMT22) | llm | 0.85 |
| key result | pre-filling runtime improvement from LoRA: 4.9x (2PC, OpenLLaMA-3B-v2 (92% of 2PC runtime in multiplications)) | llm | 0.85 |
| key result | head-merging improvement: 2.1-2.4x runtime, 2.6-2.8x communication (3PC and 2PC-Dealer, pre-filling stage) | llm | 0.85 |
| key result | MTBench score: 4.95 fully fine-tuned vs 4.77 (LF=0.5), 4.72 (LoRA=64), 4.3 (HM=4), 1.25 zero-shot (OpenLLaMA-3B-v2, Figure 5) | llm | 0.85 |
System card: MPC-minimized secure LLM inference (Marill)¶
| Group | 5. Interactive MPC and HE+MPC hybrids |
| Family | MPC + LLM fine-tuning co-design |
| Arithmetic | Fixed point |
| Interaction | Interactive 2PC/3PC |
| Model scale | Llama-2-7B |
| Status | Active SOTA |
| Relation to the SoK | Fine-tunes LLMs so fewer layers need MPC. |
| Note | Model-modifying. Combines public pre-trained layers with private ones. |
| Systems page | MPC-minimized secure LLM inference (Marill) |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: MPC 110 [p.1], Secret sharing 8 [p.4], Zero-knowledge proofs 2 [p.17]
- library: ABY / ABY3 1 [p.13], CrypTen 1 [p.12], EzPC / CrypTFlow 1 [p.13]
- model: LoRA 32 [p.3], Transformer 7 [p.1], LLaMA 6 [p.1], BERT 3 [p.12], GPT 2 [p.1], RNN / LSTM 1 [p.13]
- hardware: GPU 6 [p.3], NVIDIA V100 2 [p.7], Intel Xeon 1 [p.7]
- technique: Fine-tuning 66 [p.1], Two-party computation 49 [p.3], Pruning 21 [p.3], Three-party computation 20 [p.3], Attention 14 [p.3], ReLU 10 [p.4], Softmax 7 [p.3], Semi-honest model 7 [p.4], Malicious model 7 [p.4], GELU 3 [p.3], LayerNorm 2 [p.5], Matrix multiplication 2 [p.3]
- task: Language modeling / generation 25 [p.1], Machine translation 6 [p.3]
- topic: Private / secure inference 76 [p.1], Large language models 34 [p.1], Open source 15 [p.1], Benchmark 11 [p.3], Homomorphic encryption 2 [p.10]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/5-076_mpc-minimized-secure-llm-inference.pdf |
catalogue | 20 | 1.2 MB | 3f197aef58f6 |
Database key: mpc-minimized-secure-llm-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set mpc-minimized-secure-llm-inference COLUMN VALUE