ELLMo: Packing- and Depth-Aware Encrypted Transformer Inference¶
Authors: S. N. Guzelhan; L. Daksha; C. A. Domingo; G. Jonatan; J. Kim; J. L. Abellan; D. Kaeli; A. Joshi
Year: 2026
Venue: IACR ePrint 2026/198
type: system collection: catalogue, sok-references status: auto
Links: ePrint 2026/198 · Semantic Scholar
Summary¶
Auto-extracted summary (unreviewed)
ELLMo is a CKKS-based encrypted transformer inference design that introduces a shallow plaintext-ciphertext matrix multiplication (shifting rotations to plaintext, 28-50% fewer rotations than FIDESlib and THOR), fuses head-split/merge masking into the ciphertext-ciphertext multiply, and reduces nonlinear depth with a Statistical-max Softmax and a DelayNorm that avoid deep comparison trees and homomorphic divisions, cutting bootstrapping by up to 46%. Implemented on the GPU library FIDESlib (H200), it runs encrypted BERT-Tiny 1.4x faster than state-of-the-art baselines with 0-1.5% accuracy loss on SST-2, MRPC and RTE.
Abstract¶
Cloud-based Large Language Model (LLM) inference processes sensitive user inputs, yet current deployments offer limited confidentiality guarantees. Fully Homomorphic Encryption (FHE) can provide strong privacy, but it clashes with transformer architectures, where rigid ciphertext packing demands expensive rotations, and deep polynomial circuits for nonlinearities necessitate costly bootstrapping. Although recent work has reported promising speedups, maintaining model accuracy is a challenge. We address these issues with ELLMo, a packing- and depth-aware encrypted transformer design. ELLMo introduces a novel matrix multiplication algorithm to reduce ciphertext rotations. Further, head-split and merge steps are fused into this new algorithm at no additional cost. To reduce the depth of nonlinear layers, our contributions, Statistical-max Softmax and DelayNorm, help bypass deep comparison trees and homomorphic divisions to reduce bootstrapping by up to 46%. On encrypted BERT-Tiny, ELLMo achieves a 1.4× speedup over state-of-the-art baselines with 0%–1.5% accuracy loss across SST-2, MRPC, and RTE downstream tasks.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Shallow PCMM algorithm reducing ciphertext rotations and key-switching | llm | 0.85 |
| contribution | Head-split and merge fused into CCMM at no extra cost | llm | 0.85 |
| contribution | Statistical-max Softmax (max approximated as mu + p*sigma) avoiding comparison trees | llm | 0.85 |
| contribution | DelayNorm delayed normalization avoiding homomorphic division in LayerNorm | llm | 0.85 |
| contribution | GPU implementation on FIDESlib with 1.4x speedup on BERT-Tiny | llm | 0.85 |
| schemes | CKKS | llm | 0.85 |
| libraries | FIDESlib | llm | 0.85 |
| models | BERT-Tiny | llm | 0.85 |
| datasets | SST-2 | llm | 0.85 |
| datasets | MRPC | llm | 0.85 |
| datasets | RTE | llm | 0.85 |
| tasks | private transformer inference | llm | 0.85 |
| tasks | GLUE text classification | llm | 0.85 |
| hardware | NVIDIA H200 GPU | llm | 0.85 |
| security level | 128-bit | llm | 0.85 |
| threat model | client-server: cloud evaluates the LLM on ciphertexts, only the user can decrypt | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| limitations | Statistical-max Softmax degrades sharply when targeting more than ~95% distribution coverage (p > 2.5); accuracy depends on choosing p in the 80-95% coverage range | llm | 0.85 |
| llm relevance | high | llm | 0.85 |
| key result | speed-up: 1.4x (encrypted BERT-Tiny vs state-of-the-art baselines, FIDESlib on H200 GPU) | llm | 0.85 |
| key result | accuracy loss: 0%-1.5% (SST-2, MRPC, RTE) | llm | 0.85 |
| key result | bootstrapping reduction: up to 46% (via Statistical-max Softmax and DelayNorm) | llm | 0.85 |
| key result | rotation count reduction: 28%-50% (linear layers vs FIDESlib and THOR, BERT-Tiny and BERT-base configurations) | llm | 0.85 |
System card: ELLMO¶
| Group | 0. CKKS LLM frameworks surveyed in the SoK |
| Family | CKKS · Model-modifying (N-MOD) |
| Arithmetic | Approximate real (CKKS) |
| Interaction | non-interactive FHE |
| Model scale | BERT-Tiny |
| Library | OpenFHE + FIDESlib |
| Hardware | H200 GPU |
| Packing | L-ALG |
| Latency | 8 s unamortized |
| Open source | partial |
| Status | Active SOTA |
| Relation to the SoK | Pre-computed plaintext transforms to minimize depth; delayed LayerNorm; statistical max estimation. |
| Note | Fastest BERT-Tiny entry in the SoK; sequence length not reported. |
| Systems page | ELLMO |
SoK reference [77]¶
S. N. Guzelhan, L. Daksha, C. A. Domingo, G. Jonatan, J. Kim, J. L. Abellan, D. Kaeli, and A. Joshi, “Ellmo: Packing-and depthaware encrypted transformer inference,” Cryptology ePrint Archive, 2026.
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 25 [p.1], BFV 3 [p.3]
- library: FIDESlib 13 [p.3], Microsoft SEAL 3 [p.13], Orion 2 [p.13], Phantom 1 [p.16], HEonGPU 1 [p.16], Cheddar 1 [p.14], HEIR 1 [p.13], Fhelipe 1 [p.13], PyTorch 1 [p.13]
- model: BERT 39 [p.1], Transformer 7 [p.1], Logistic regression 2 [p.8], GPT 1 [p.12], LLaMA 1 [p.12], CNN 1 [p.10]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 28 [p.1], SST-2 16 [p.1], GLUE 6 [p.2]
- hardware: GPU 11 [p.3], Multi-GPU 2 [p.13], FPGA 1 [p.13]
- technique: Rotations 38 [p.1], Softmax 34 [p.1], LayerNorm 27 [p.1], Bootstrapping 20 [p.1], Matrix multiplication 20 [p.1], Key switching 15 [p.2], Chebyshev 14 [p.4], GELU 13 [p.1], Polynomial approximation 6 [p.1], Rescaling 5 [p.3], RNS 5 [p.3], Compiler 5 [p.13], Attention 3 [p.2], Relinearization 2 [p.3], Newton iteration 2 [p.9], SIMD packing 2 [p.1], Convolution 2 [p.8], Two-party computation 2 [p.5]
- task: Medical / healthcare 2 [p.1], Finance 2 [p.1]
- topic: Homomorphic encryption 60 [p.1], Large language models 17 [p.1], Benchmark 5 [p.11], Private / secure inference 4 [p.1], Open source 4 [p.3]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/0-007_ellmo-packing-and-depth-aware-encrypted-transformer-inferenc.pdf |
catalogue | 17 | 720.9 KB | 40ca842a6358 |
|
references/sok-references/077_ellmo-packing-and-depth-aware-encrypted-transformer-inferenc.pdf |
sok-references | 17 | 720.9 KB | 40ca842a6358 |
byte-identical to references/catalogue/0-007_ellmo-packing-and-depth-aware-encrypted-transformer-inferenc.pdf |
Database key: ellmo-packing-and-depth-aware-encrypted-transformer-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set ellmo-packing-and-depth-aware-encrypted-transformer-inference COLUMN VALUE