Skip to content

EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption

Authors: L. de Castro; A. Polychroniadou; D. Escudero
Year: 2024
Venue: NeurIPS 2024 workshop; ICML 2025 (EncryptedLLM)

type: system collection: catalogue, sok-references status: auto

Links: link

Summary

Auto-extracted summary (unreviewed)

Proposes a GPU-accelerated FHE implementation (an open-sourced OpenFHE GPU extension) and uses it to benchmark a fully encrypted GPT-2 (small) forward pass, generating a token at position 128 without any client interaction, at over 200x the speed of the CPU baseline. It also studies polynomial/iterative approximations of GeLU, LayerNorm (Newton iterations) and Softmax (exp plus Goldschmidt division) and shows via lm-evaluation-harness on eight tasks that GPT-2 small/medium/large lose little accuracy under these approximations.

Abstract

As large language models (LLMs) become more powerful, the computation required to run these models is increasingly outsourced to a thirdparty cloud. While this saves clients’ computation, it risks leaking the clients’ LLM queries to the cloud provider. Fully homomorphic encryption (FHE) presents a natural solution to this problem: simply encrypt the query and evaluate the LLM homomorphically on the cloud machine. The result remains encrypted and can only be learned by the client who holds the secret key. In this work, we propose a GPU-accelerated FHE scheme and leverage it to benchmark an encrypted GPT-2 forward pass. Our approach achieves runtimes that are over 200× faster than the CPU baseline. We also present novel and extensive experimental analysis of approximations of LLM activation functions to maintain accuracy while achieving this performance.
Source: pdf

Extracted details

Field Value Source Confidence
contribution GPU-accelerated FHE scheme (OpenFHE+GPU extension, open-sourced) llm 0.75
contribution End-to-end encrypted GPT-2 forward pass benchmark, >200x faster than CPU llm 0.75
contribution FHE-compatible approximations of GeLU, LayerNorm and Softmax with accuracy study on standard LLM benchmarks llm 0.75
contribution Non-interactive autoregressive generation: output can be fed back for the next token without client interaction llm 0.75
schemes FHE with bootstrapping (OpenFHE-based) llm 0.75
libraries OpenFHE llm 0.75
libraries HuggingFace Transformers llm 0.75
libraries lm-evaluation-harness llm 0.75
models GPT-2 small llm 0.75
models GPT-2 medium llm 0.75
models GPT-2 large llm 0.75
datasets HellaSwag llm 0.75
datasets ARC-Easy llm 0.75
datasets PIQA llm 0.75
datasets Social IQa llm 0.75
datasets MNLI llm 0.75
datasets SST-2 llm 0.75
datasets ANLI llm 0.75
datasets WiC llm 0.75
tasks private LLM inference llm 0.75
tasks language model benchmark evaluation llm 0.75
hardware GPU (model unspecified in excerpt) llm 0.75
security level 128-bit (benchmarks also at 80-bit) llm 0.75
threat model client-server: server runs the LLM on the encrypted query and never sees plaintext or intermediates llm 0.75
interaction non-interactive FHE llm 0.75
code url https://github.com/leodec/openfhe-gpu-p (URL found in the full text) regex 0.50
code url https://github.com/huggingface/tra (URL found in the full text) regex 0.50
code url https://github.com/scale-snu/ckks-gpu-c (URL found in the full text) regex 0.50
code url https://github.com/leodec/openfhe-gpu-p llm 0.75
llm relevance high llm 0.75
key result speed-up: >200x over CPU baseline (encrypted GPT-2 small forward pass, GPU-accelerated FHE) llm 0.75
key result accuracy: little degradation vs baseline (GPT-2 small/medium/large on 8 lm-eval-harness tasks with approximated GeLU/LayerNorm/Softmax) llm 0.75

System card: CEP / EncryptedLLM (GPU-accelerated FHE GPT-2)

Group 0. CKKS LLM frameworks surveyed in the SoK
Family CKKS · Model-preserving (N-PRE)
Arithmetic Approximate real (CKKS)
Interaction non-interactive FHE
Model scale GPT-2
Library Custom OpenFHE GPU backend
Hardware A100 GPU
Packing L-ROW
Latency ~100 s per generated token
Open source no
Status Active SOTA
Relation to the SoK First open GPU-accelerated CKKS bootstrapping backend for OpenFHE; model-preserving GPT-2 decoding.
Note ~150x speedup over CPU OpenFHE; adopts NEXUS arg max.
Systems page CEP / EncryptedLLM (GPU-accelerated FHE GPT-2)

SoK reference [17]

L. de Castro, A. Polychroniadou, and D. Escudero, “Privacypreserving large language model inference via gpu-accelerated fully homomorphic encryption,” in Neurips Safe Generative AI Workshop 2024, 2024.

Listed on the reference page of the site.

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 16 [p.2], MPC 5 [p.3]
  • library: OpenFHE 12 [p.2], HuggingFace 3 [p.3], Microsoft SEAL 2 [p.2]
  • model: GPT 45 [p.1], LLaMA 3 [p.1], CNN 2 [p.4], Logistic regression 2 [p.2]
  • dataset: SST-2 3 [p.7], GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 3 [p.7], ARC 3 [p.7], HellaSwag 2 [p.7]
  • hardware: GPU 39 [p.1], NVIDIA A100 2 [p.8], Intel Xeon 2 [p.8]
  • technique: Bootstrapping 23 [p.3], Softmax 23 [p.3], LayerNorm 20 [p.3], GELU 14 [p.3], Polynomial approximation 5 [p.2], Taylor expansion 5 [p.6], Goldschmidt 4 [p.6], Newton iteration 2 [p.7], Quantization 2 [p.2]
  • task: Medical / healthcare 2 [p.10]
  • topic: Large language models 84 [p.1], Homomorphic encryption 82 [p.1], Benchmark 32 [p.1], Open source 10 [p.2], Private / secure inference 4 [p.3]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/0-002_encryptedllm-privacy-preserving-large-language-model-inferen.pdf catalogue 12 1.9 MB fd60fc2b5542
references/sok-references/017_privacy-preserving-large-language-model-inference-via-gpu-ac.pdf sok-references 12 1.9 MB fd60fc2b5542 byte-identical to references/catalogue/0-002_encryptedllm-privacy-preserving-large-language-model-inferen.pdf

Database key: encryptedllm-privacy-preserving-large-language-model-inference-via-gpu · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set encryptedllm-privacy-preserving-large-language-model-inference-via-gpu COLUMN VALUE