Skip to content

SoK: Private LLM Inference using Approximate Homomorphic Encryption

Authors: Al Badawi, Alexandru, Polyakov, Vaikuntanathan
Year: 2026
Venue: SoK paper (IACR ePrint 2026/935)

type: system collection: sok, catalogue status: auto

Links: ePrint 2026/935

Summary

Auto-extracted summary (unreviewed)

This SoK is the first systematization of non-interactive CKKS-based private LLM inference, organizing 20 frameworks along a model-level axis (packing layouts for linear blocks, model preservation vs. modification for non-linear blocks) and a system-level axis (hardware, compilers, bootstrapping management, hybrid execution). It proposes the Private LLM Card System (PLCS) for standardized reporting and introduces POLARIS, a model-preserving, GPU-accelerated open-source reference framework currently supporting BERT-Tiny and BERT-Mini. The analysis finds that only 20% of surveyed implementations are model-preserving, that model-preserving CKKS approximations keep high downstream accuracy from BERT-Tiny up to Llama-3-8B, and that a roughly four-orders-of-magnitude runtime gap versus plaintext inference remains the main barrier to practical use.

Abstract

Although recent surveys on privacy-enhancing technologies concluded that FHE cannot feasibly evaluate nonlinear activation functions in modern ML architectures, 20 CKKS-based frameworks have since demonstrated end-toend private inference of LLMs with up to 8 B parameters. However, as the field grows rapidly, the literature has become fragmented. Frameworks differ in ciphertext packing layouts, model fidelity, software and hardware stacks, and reported metrics, which hinder direct comparison and reproducibility. This paper presents the first systematization of knowledge for non-interactive, CKKS-based private LLM inference. We categorize the design space across two axes: a model-level axis (packing layout for linear blocks and model preservation for non-linear blocks) and a system-level axis (covering hardware, compilers, bootstrapping management, and hybrid execution). To standardize reporting framework configurations and results among researchers, we propose a Private LLM Card System (PLCS). Furthermore, we introduce POLARIS, a modelpreserving reference framework for CKKS-based private LLM inference and provide it as an open-source proof-of-concept1. In its current version, POLARIS supports encrypted inference for BERT-Tiny and BERT-Mini and leverages GPU acceleration for enhanced performance. Our analysis suggests that only 20% of surveyed implementations are model-preserving, that is, they evaluate standard, unmodified LLMs without retraining or architectural substitutions while keeping the entire prompt encrypted. We also show that model-preserving CKKS approximations maintain high downstream task accuracy from BERT-Tiny up to Llama- 3-8 B. We identify a runtime gap of roughly four orders of magnitude between encrypted and plaintext inference as the primary barrier to practical use. While our analysis confirms that CKKS-based inference is now algorithmically feasible for non-linear functions and large models, we conclude that it remains operationally impractical for many human-facing applications until the efficiency gap is narrowed. We outline promising research directions to close this gap.
Source: pdf

Extracted details

Field Value Source Confidence
contribution First systematization of knowledge for non-interactive CKKS-based private LLM inference (20 frameworks) llm 0.90
contribution Two-axis design-space taxonomy: model-level (packing layout, model preservation) and system-level (hardware, compilers, bootstrapping, hybrid execution) llm 0.90
contribution Private LLM Card System (PLCS): a four-card reporting scheme for model, cryptographic, system and application configuration llm 0.90
contribution POLARIS: open-source, model-preserving, GPU-accelerated reference framework for CKKS-based private LLM inference llm 0.90
contribution First open-source GPU-accelerated backend for CKKS bootstrapping in OpenFHE llm 0.90
contribution Identification of the ~10^4x runtime gap and research directions to close it llm 0.90
schemes CKKS llm 0.90
libraries OpenFHE llm 0.90
libraries SEAL llm 0.90
libraries Lattigo llm 0.90
libraries HEaaN llm 0.90
libraries Phantom llm 0.90
libraries FIDESlib llm 0.90
libraries Liberate llm 0.90
models BERT-Tiny llm 0.90
models BERT-Mini llm 0.90
models Llama-3-8B llm 0.90
tasks private LLM inference llm 0.90
tasks encrypted text classification llm 0.90
tasks encrypted token generation llm 0.90
hardware GPU llm 0.90
security level 128-bit llm 0.90
threat model client-server outsourced inference with the prompt encrypted end-to-end (non-interactive) llm 0.90
interaction non-interactive FHE llm 0.90
code url https://github.com/tuneinsight/lattigo (URL found in the full text) regex 0.50
code url https://github.com/Desilo/liberate-fhe (URL found in the full text) regex 0.50
code url https://github.com/bytedance/ByteTransformer (URL found in the full text) regex 0.50
limitations POLARIS currently supports only BERT-Tiny and BERT-Mini llm 0.90
limitations CKKS-based LLM inference remains operationally impractical for many human-facing applications because of the ~10^4x runtime gap llm 0.90
limitations POLARIS code is to be released only upon formal publication llm 0.90
llm relevance high llm 0.90
key result share of model-preserving frameworks: 20% (20 surveyed CKKS-based private LLM inference frameworks) llm 0.90
key result runtime gap, encrypted vs plaintext inference: ~4 orders of magnitude (comparable system configurations) llm 0.90
key result largest model with end-to-end CKKS private inference: 8B parameters (Llama-3-8B) (surveyed frameworks, high downstream task accuracy maintained) llm 0.90

System card: POLARIS

Group 0. CKKS LLM frameworks surveyed in the SoK
Family CKKS · Model-preserving (N-PRE)
Arithmetic Approximate real (CKKS)
Interaction non-interactive FHE
Model scale BERT-Tiny, BERT-Mini
Library OpenFHE + FIDESlib
Hardware A100 GPU
Packing L-ROW
Latency 18 s (BERT-Tiny), 83 s (BERT-Mini), unamortized
Open source promised at publication
Status Active SOTA
Relation to the SoK Reference CKKS implementation combining the SoK's recommended model-preserving building blocks.
Note Open-source PoC reference framework; model-preserving, GPU accelerated; 0% accuracy loss on SST-2 subset.
Systems page POLARIS

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 116 [p.1], MPC 5 [p.1], Matrix-native FHE 3 [p.18], PIR 3 [p.5], BFV 2 [p.7], BGV 2 [p.7], TEE / confidential computing 2 [p.1]
  • library: OpenFHE 21 [p.2], Phantom 17 [p.11], Microsoft SEAL 15 [p.2], Lattigo 8 [p.2], Liberate.FHE 6 [p.2], HEaaN (library) 3 [p.2], HELayers 3 [p.13], FIDESlib 2 [p.2], Orion 2 [p.10], HElib 1 [p.21], Concrete (Zama) 1 [p.12]
  • model: BERT 60 [p.1], LLaMA 47 [p.1], GPT 21 [p.1], Transformer 15 [p.1], LoRA 3 [p.11], SVM 1 [p.19]
  • dataset: GLUE 14 [p.5], SST-2 10 [p.12], GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 5 [p.5], HellaSwag 2 [p.5], ARC 2 [p.5], SQuAD 1 [p.20], LAMBADA 1 [p.20]
  • hardware: GPU 77 [p.1], NVIDIA A100 28 [p.11], NVIDIA RTX 8 [p.14], Multi-GPU 8 [p.1], NVIDIA H100 4 [p.17], Intel Xeon 1 [p.15], CUDA 1 [p.15]
  • technique: Bootstrapping 57 [p.1], Softmax 30 [p.1], Matrix multiplication 25 [p.3], LayerNorm 22 [p.4], Rotations 16 [p.6], Non-interactive 14 [p.1], GELU 12 [p.2], SIMD packing 11 [p.1], Polynomial approximation 9 [p.2], Fine-tuning 7 [p.2], KV cache 7 [p.8], Compiler 7 [p.3], Chebyshev 5 [p.7], Sigmoid 5 [p.7], Key switching 4 [p.6], Rescaling 4 [p.6], Attention 4 [p.3], Relinearization 3 [p.6], Goldschmidt 3 [p.9], RNS 3 [p.6], Functional bootstrapping 2 [p.18], Minimax approximation 2 [p.7], Semi-honest model 2 [p.6]
  • assumption: LWE 3 [p.7]
  • task: Language modeling / generation 13 [p.4], Medical / healthcare 3 [p.1], Question answering 2 [p.5], Finance 2 [p.1]
  • topic: Large language models 107 [p.1], Homomorphic encryption 72 [p.1], Open source 21 [p.1], Benchmark 18 [p.2], Private / secure inference 13 [p.1], Privacy-preserving machine learning 4 [p.2]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
2026-935.pdf sok 22 763.4 KB fafec106d876
references/catalogue/0-000_sok-private-llm-inference-using-approximate-homomorphic-encr.pdf catalogue 22 763.7 KB 9daf3c572b5f

Database key: sok-private-llm-inference-using-approximate-homomorphic-encryption · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set sok-private-llm-inference-using-approximate-homomorphic-encryption COLUMN VALUE