Encryption-friendly LLM architecture¶
Authors: Donghwan Rho; Taeseong Kim; Minje Park; Jung Woo Kim; Hyunsik Chae; Ernest K. Ryu; Jung Hee Cheon
Year: 2025
Venue: ICLR 2025
type: system collection: catalogue, sok-references status: auto
Links: arXiv:2410.02486
Summary¶
Auto-extracted summary (unreviewed)
Proposes an HE-friendly transformer architecture aimed at private fine-tuning followed by inference under CKKS: LoRA fine-tuning keeps most matrix products plaintext-ciphertext (PCMM is ~4.6x faster than CCMM), and a Gaussian-kernel (GK) attention replaces softmax (6.24x faster than the HETAL softmax approximation). Implemented with HEaaN on 8 RTX 4090 GPUs for a 2-layer BERT-style encoder, the design yields 6.94x faster fine-tuning and 2.3x faster inference while keeping performance comparable to plaintext models.
Abstract¶
Large language models (LLMs) offer personalized responses based on user interactions, but this use case raises serious privacy concerns. Homomorphic encryption (HE) is a cryptographic protocol supporting arithmetic computations in encrypted states and provides a potential solution for privacy-preserving machine learning (PPML). However, the computational intensity of transformers poses challenges for applying HE to LLMs. In this work, we propose a modified HE-friendly transformer architecture with an emphasis on inference following personalized (private) fine-tuning. Utilizing LoRA fine-tuning and Gaussian kernels, we achieve significant computational speedups -- 6.94x for fine-tuning and 2.3x for inference -- while maintaining performance comparable to plaintext models. Our findings provide a viable proof of concept for offering privacy-preserving LLM services in areas where data protection is crucial. Our code is available on GitHub.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | HE-friendly transformer architecture for private LoRA fine-tuning and inference | llm | 0.90 |
| contribution | LoRA fine-tuning to replace ciphertext-ciphertext matmuls with plaintext-ciphertext ones | llm | 0.90 |
| contribution | Gaussian kernel attention replacing softmax under HE | llm | 0.90 |
| contribution | AdamW-HE optimizer variant that avoids numerically hard operations | llm | 0.90 |
| contribution | HEaaN GPU implementation with open-source code | llm | 0.90 |
| schemes | CKKS | llm | 0.90 |
| libraries | HEaaN | llm | 0.90 |
| libraries | Sollya | llm | 0.90 |
| models | BERT-style encoder (2 layers, 768 hidden, 12 heads, LoRA rank 2) | llm | 0.90 |
| tasks | private LLM fine-tuning | llm | 0.90 |
| tasks | private transformer inference | llm | 0.90 |
| tasks | text classification | llm | 0.90 |
| hardware | 8x NVIDIA GeForce RTX 4090 24GB | llm | 0.90 |
| interaction | non-interactive FHE | llm | 0.90 |
| code url | https://github.com/Donghwan-Rho/Encryption-friendly_LLM_Architecture (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/Donghwan-Rho/Encryption-friendly_LLM_Architecture | llm | 0.90 |
| limitations | Transformer limited to 2 layers for practical computation time | llm | 0.90 |
| llm relevance | high | llm | 0.90 |
| key result | fine-tuning speed-up: 6.94x (HE-friendly architecture vs baseline under HE) | llm | 0.90 |
| key result | inference speed-up: 2.3x (HE-friendly architecture vs baseline under HE, 128 tokens) | llm | 0.90 |
| key result | PCMM vs CCMM time: 0.275 s vs 1.259 s (4.58x) (128x768 by 768x768 matmul, HEaaN) | llm | 0.90 |
| key result | GK vs Softmax time: 1.44 s vs 8.99 s (6.24x) (12 pairs of 128x128 matrices under HE) | llm | 0.90 |
System card: EFLA (Encryption-Friendly LLM Architecture)¶
| Group | 0. CKKS LLM frameworks surveyed in the SoK |
| Family | CKKS · Model-modifying (N-MOD) |
| Arithmetic | Approximate real (CKKS) |
| Interaction | non-interactive FHE |
| Model scale | BERT-like (fine-tuning + inference) |
| Library | HEaaN |
| Hardware | RTX 4090 GPU |
| Packing | L-ROW |
| Latency | 27 s per layer |
| Open source | yes |
| Status | Active SOTA |
| Relation to the SoK | Gaussian-kernel attention replaces Softmax; LoRA fine-tuning under CKKS. |
| Note | Also addresses encrypted fine-tuning, not just inference. |
| Code | https://github.com/CryptoLabInc/EFLA |
| Systems page | EFLA (Encryption-Friendly LLM Architecture) |
SoK reference [80]¶
D. Rho, T. Kim, M. Park, J. W. Kim, H. Chae, E. K. Ryu, and J. H. Cheon, “Encryption-friendly LLM architecture,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=pbre0HKsfE
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 15 [p.1], MPC 9 [p.1], Differential privacy 2 [p.1]
- library: HEaaN (library) 5 [p.7], HElib 1 [p.11]
- model: LoRA 83 [p.1], BERT 19 [p.1], LLaMA 6 [p.2], GPT 5 [p.1], Transformer 3 [p.2], CNN 3 [p.10], Logistic regression 1 [p.12]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 26 [p.9], SST-2 7 [p.9], GLUE 3 [p.9]
- hardware: GPU 10 [p.7], NVIDIA RTX 1 [p.7], Multi-GPU 1 [p.24]
- technique: Fine-tuning 69 [p.1], Matrix multiplication 48 [p.2], Softmax 47 [p.2], ReLU 19 [p.7], Polynomial approximation 13 [p.3], Attention 11 [p.2], Remez 9 [p.7], LayerNorm 8 [p.10], Bootstrapping 7 [p.4], Key switching 7 [p.4], Rotations 6 [p.4], Non-interactive 6 [p.3], Convolution 4 [p.11], Rescaling 3 [p.19], Minimax approximation 3 [p.7], SIMD packing 2 [p.4]
- assumption: Lattice-based 3 [p.11], RLWE 2 [p.17], LWE 2 [p.13]
- task: Language modeling / generation 4 [p.11], Text classification 2 [p.10], Machine translation 2 [p.12]
- topic: Homomorphic encryption 122 [p.1], Large language models 27 [p.1], Privacy-preserving machine learning 7 [p.1], Private / secure inference 6 [p.3], Benchmark 2 [p.9]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/0-010_encryption-friendly-llm-architecture.pdf |
catalogue | 27 | 1.4 MB | efe43408cc9c |
|
references/sok-references/080_encryption-friendly-llm-architecture.pdf |
sok-references | 27 | 1.4 MB | efe43408cc9c |
byte-identical to references/catalogue/0-010_encryption-friendly-llm-architecture.pdf |
Database key: encryption-friendly-llm-architecture · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set encryption-friendly-llm-architecture COLUMN VALUE