EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption¶
Authors: L. de Castro; A. Polychroniadou; D. Escudero
Year: 2024
Venue: NeurIPS 2024 workshop; ICML 2025 (EncryptedLLM)
type: system collection: catalogue, sok-references status: auto
Links: link
Summary¶
Auto-extracted summary (unreviewed)
Proposes a GPU-accelerated FHE implementation (an open-sourced OpenFHE GPU extension) and uses it to benchmark a fully encrypted GPT-2 (small) forward pass, generating a token at position 128 without any client interaction, at over 200x the speed of the CPU baseline. It also studies polynomial/iterative approximations of GeLU, LayerNorm (Newton iterations) and Softmax (exp plus Goldschmidt division) and shows via lm-evaluation-harness on eight tasks that GPT-2 small/medium/large lose little accuracy under these approximations.
Abstract¶
As large language models (LLMs) become more powerful, the computation required to run these models is increasingly outsourced to a thirdparty cloud. While this saves clients’ computation, it risks leaking the clients’ LLM queries to the cloud provider. Fully homomorphic encryption (FHE) presents a natural solution to this problem: simply encrypt the query and evaluate the LLM homomorphically on the cloud machine. The result remains encrypted and can only be learned by the client who holds the secret key. In this work, we propose a GPU-accelerated FHE scheme and leverage it to benchmark an encrypted GPT-2 forward pass. Our approach achieves runtimes that are over 200× faster than the CPU baseline. We also present novel and extensive experimental analysis of approximations of LLM activation functions to maintain accuracy while achieving this performance.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | GPU-accelerated FHE scheme (OpenFHE+GPU extension, open-sourced) | llm | 0.75 |
| contribution | End-to-end encrypted GPT-2 forward pass benchmark, >200x faster than CPU | llm | 0.75 |
| contribution | FHE-compatible approximations of GeLU, LayerNorm and Softmax with accuracy study on standard LLM benchmarks | llm | 0.75 |
| contribution | Non-interactive autoregressive generation: output can be fed back for the next token without client interaction | llm | 0.75 |
| schemes | FHE with bootstrapping (OpenFHE-based) | llm | 0.75 |
| libraries | OpenFHE | llm | 0.75 |
| libraries | HuggingFace Transformers | llm | 0.75 |
| libraries | lm-evaluation-harness | llm | 0.75 |
| models | GPT-2 small | llm | 0.75 |
| models | GPT-2 medium | llm | 0.75 |
| models | GPT-2 large | llm | 0.75 |
| datasets | HellaSwag | llm | 0.75 |
| datasets | ARC-Easy | llm | 0.75 |
| datasets | PIQA | llm | 0.75 |
| datasets | Social IQa | llm | 0.75 |
| datasets | MNLI | llm | 0.75 |
| datasets | SST-2 | llm | 0.75 |
| datasets | ANLI | llm | 0.75 |
| datasets | WiC | llm | 0.75 |
| tasks | private LLM inference | llm | 0.75 |
| tasks | language model benchmark evaluation | llm | 0.75 |
| hardware | GPU (model unspecified in excerpt) | llm | 0.75 |
| security level | 128-bit (benchmarks also at 80-bit) | llm | 0.75 |
| threat model | client-server: server runs the LLM on the encrypted query and never sees plaintext or intermediates | llm | 0.75 |
| interaction | non-interactive FHE | llm | 0.75 |
| code url | https://github.com/leodec/openfhe-gpu-p (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/huggingface/tra (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/scale-snu/ckks-gpu-c (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/leodec/openfhe-gpu-p | llm | 0.75 |
| llm relevance | high | llm | 0.75 |
| key result | speed-up: >200x over CPU baseline (encrypted GPT-2 small forward pass, GPU-accelerated FHE) | llm | 0.75 |
| key result | accuracy: little degradation vs baseline (GPT-2 small/medium/large on 8 lm-eval-harness tasks with approximated GeLU/LayerNorm/Softmax) | llm | 0.75 |
System card: CEP / EncryptedLLM (GPU-accelerated FHE GPT-2)¶
| Group | 0. CKKS LLM frameworks surveyed in the SoK |
| Family | CKKS · Model-preserving (N-PRE) |
| Arithmetic | Approximate real (CKKS) |
| Interaction | non-interactive FHE |
| Model scale | GPT-2 |
| Library | Custom OpenFHE GPU backend |
| Hardware | A100 GPU |
| Packing | L-ROW |
| Latency | ~100 s per generated token |
| Open source | no |
| Status | Active SOTA |
| Relation to the SoK | First open GPU-accelerated CKKS bootstrapping backend for OpenFHE; model-preserving GPT-2 decoding. |
| Note | ~150x speedup over CPU OpenFHE; adopts NEXUS arg max. |
| Systems page | CEP / EncryptedLLM (GPU-accelerated FHE GPT-2) |
SoK reference [17]¶
L. de Castro, A. Polychroniadou, and D. Escudero, “Privacypreserving large language model inference via gpu-accelerated fully homomorphic encryption,” in Neurips Safe Generative AI Workshop 2024, 2024.
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 16 [p.2], MPC 5 [p.3]
- library: OpenFHE 12 [p.2], HuggingFace 3 [p.3], Microsoft SEAL 2 [p.2]
- model: GPT 45 [p.1], LLaMA 3 [p.1], CNN 2 [p.4], Logistic regression 2 [p.2]
- dataset: SST-2 3 [p.7], GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 3 [p.7], ARC 3 [p.7], HellaSwag 2 [p.7]
- hardware: GPU 39 [p.1], NVIDIA A100 2 [p.8], Intel Xeon 2 [p.8]
- technique: Bootstrapping 23 [p.3], Softmax 23 [p.3], LayerNorm 20 [p.3], GELU 14 [p.3], Polynomial approximation 5 [p.2], Taylor expansion 5 [p.6], Goldschmidt 4 [p.6], Newton iteration 2 [p.7], Quantization 2 [p.2]
- task: Medical / healthcare 2 [p.10]
- topic: Large language models 84 [p.1], Homomorphic encryption 82 [p.1], Benchmark 32 [p.1], Open source 10 [p.2], Private / secure inference 4 [p.3]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/0-002_encryptedllm-privacy-preserving-large-language-model-inferen.pdf |
catalogue | 12 | 1.9 MB | fd60fc2b5542 |
|
references/sok-references/017_privacy-preserving-large-language-model-inference-via-gpu-ac.pdf |
sok-references | 12 | 1.9 MB | fd60fc2b5542 |
byte-identical to references/catalogue/0-002_encryptedllm-privacy-preserving-large-language-model-inferen.pdf |
Database key: encryptedllm-privacy-preserving-large-language-model-inference-via-gpu · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set encryptedllm-privacy-preserving-large-language-model-inference-via-gpu COLUMN VALUE