Skip to content

Encryption-friendly LLM architecture

Authors: Donghwan Rho; Taeseong Kim; Minje Park; Jung Woo Kim; Hyunsik Chae; Ernest K. Ryu; Jung Hee Cheon
Year: 2025
Venue: ICLR 2025

type: system collection: catalogue, sok-references status: auto

Links: arXiv:2410.02486

Summary

Auto-extracted summary (unreviewed)

Proposes an HE-friendly transformer architecture aimed at private fine-tuning followed by inference under CKKS: LoRA fine-tuning keeps most matrix products plaintext-ciphertext (PCMM is ~4.6x faster than CCMM), and a Gaussian-kernel (GK) attention replaces softmax (6.24x faster than the HETAL softmax approximation). Implemented with HEaaN on 8 RTX 4090 GPUs for a 2-layer BERT-style encoder, the design yields 6.94x faster fine-tuning and 2.3x faster inference while keeping performance comparable to plaintext models.

Abstract

Large language models (LLMs) offer personalized responses based on user interactions, but this use case raises serious privacy concerns. Homomorphic encryption (HE) is a cryptographic protocol supporting arithmetic computations in encrypted states and provides a potential solution for privacy-preserving machine learning (PPML). However, the computational intensity of transformers poses challenges for applying HE to LLMs. In this work, we propose a modified HE-friendly transformer architecture with an emphasis on inference following personalized (private) fine-tuning. Utilizing LoRA fine-tuning and Gaussian kernels, we achieve significant computational speedups -- 6.94x for fine-tuning and 2.3x for inference -- while maintaining performance comparable to plaintext models. Our findings provide a viable proof of concept for offering privacy-preserving LLM services in areas where data protection is crucial. Our code is available on GitHub.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution HE-friendly transformer architecture for private LoRA fine-tuning and inference llm 0.90
contribution LoRA fine-tuning to replace ciphertext-ciphertext matmuls with plaintext-ciphertext ones llm 0.90
contribution Gaussian kernel attention replacing softmax under HE llm 0.90
contribution AdamW-HE optimizer variant that avoids numerically hard operations llm 0.90
contribution HEaaN GPU implementation with open-source code llm 0.90
schemes CKKS llm 0.90
libraries HEaaN llm 0.90
libraries Sollya llm 0.90
models BERT-style encoder (2 layers, 768 hidden, 12 heads, LoRA rank 2) llm 0.90
tasks private LLM fine-tuning llm 0.90
tasks private transformer inference llm 0.90
tasks text classification llm 0.90
hardware 8x NVIDIA GeForce RTX 4090 24GB llm 0.90
interaction non-interactive FHE llm 0.90
code url https://github.com/Donghwan-Rho/Encryption-friendly_LLM_Architecture (URL found in the full text) regex 0.50
code url https://github.com/Donghwan-Rho/Encryption-friendly_LLM_Architecture llm 0.90
limitations Transformer limited to 2 layers for practical computation time llm 0.90
llm relevance high llm 0.90
key result fine-tuning speed-up: 6.94x (HE-friendly architecture vs baseline under HE) llm 0.90
key result inference speed-up: 2.3x (HE-friendly architecture vs baseline under HE, 128 tokens) llm 0.90
key result PCMM vs CCMM time: 0.275 s vs 1.259 s (4.58x) (128x768 by 768x768 matmul, HEaaN) llm 0.90
key result GK vs Softmax time: 1.44 s vs 8.99 s (6.24x) (12 pairs of 128x128 matrices under HE) llm 0.90

System card: EFLA (Encryption-Friendly LLM Architecture)

Group 0. CKKS LLM frameworks surveyed in the SoK
Family CKKS · Model-modifying (N-MOD)
Arithmetic Approximate real (CKKS)
Interaction non-interactive FHE
Model scale BERT-like (fine-tuning + inference)
Library HEaaN
Hardware RTX 4090 GPU
Packing L-ROW
Latency 27 s per layer
Open source yes
Status Active SOTA
Relation to the SoK Gaussian-kernel attention replaces Softmax; LoRA fine-tuning under CKKS.
Note Also addresses encrypted fine-tuning, not just inference.
Code https://github.com/CryptoLabInc/EFLA
Systems page EFLA (Encryption-Friendly LLM Architecture)

SoK reference [80]

D. Rho, T. Kim, M. Park, J. W. Kim, H. Chae, E. K. Ryu, and J. H. Cheon, “Encryption-friendly LLM architecture,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=pbre0HKsfE

Listed on the reference page of the site.

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 15 [p.1], MPC 9 [p.1], Differential privacy 2 [p.1]
  • library: HEaaN (library) 5 [p.7], HElib 1 [p.11]
  • model: LoRA 83 [p.1], BERT 19 [p.1], LLaMA 6 [p.2], GPT 5 [p.1], Transformer 3 [p.2], CNN 3 [p.10], Logistic regression 1 [p.12]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 26 [p.9], SST-2 7 [p.9], GLUE 3 [p.9]
  • hardware: GPU 10 [p.7], NVIDIA RTX 1 [p.7], Multi-GPU 1 [p.24]
  • technique: Fine-tuning 69 [p.1], Matrix multiplication 48 [p.2], Softmax 47 [p.2], ReLU 19 [p.7], Polynomial approximation 13 [p.3], Attention 11 [p.2], Remez 9 [p.7], LayerNorm 8 [p.10], Bootstrapping 7 [p.4], Key switching 7 [p.4], Rotations 6 [p.4], Non-interactive 6 [p.3], Convolution 4 [p.11], Rescaling 3 [p.19], Minimax approximation 3 [p.7], SIMD packing 2 [p.4]
  • assumption: Lattice-based 3 [p.11], RLWE 2 [p.17], LWE 2 [p.13]
  • task: Language modeling / generation 4 [p.11], Text classification 2 [p.10], Machine translation 2 [p.12]
  • topic: Homomorphic encryption 122 [p.1], Large language models 27 [p.1], Privacy-preserving machine learning 7 [p.1], Private / secure inference 6 [p.3], Benchmark 2 [p.9]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/0-010_encryption-friendly-llm-architecture.pdf catalogue 27 1.4 MB efe43408cc9c
references/sok-references/080_encryption-friendly-llm-architecture.pdf sok-references 27 1.4 MB efe43408cc9c byte-identical to references/catalogue/0-010_encryption-friendly-llm-architecture.pdf

Database key: encryption-friendly-llm-architecture · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set encryption-friendly-llm-architecture COLUMN VALUE