THEMIS: A Co-Designed System for Encrypted Transformer Inference¶
Authors: Li, Wang, Li, Tang, Liu, Chen
Year: 2026
Venue: IACR ePrint 2026/2264
type: system collection: catalogue status: auto
Links: ePrint 2026/2264
Summary¶
Auto-extracted summary (unreviewed)
THEMIS is a non-interactive CKKS system for encrypted transformer inference co-designed around a shared level budget: two principles bound the depth each stage may spend and fix where on the modulus chain it runs. It generalizes coefficient-encoded plaintext-ciphertext matrix multiplication to slot-encoded ciphertexts with multi-batch packing, adds two attention CCMM kernels that use the idle imaginary lane plus BSGS and hoisting, and conditions the softmax denominator with a calibrated row-wise offset. On a 52-core Xeon CPU with OpenFHE it serves BERT-base in 400.06 amortized seconds per input with GLUE accuracy comparable to plaintext, speeding PCMM 21.66x and attention CCMMs up to 7.39x over MOAI and softmax 2.57x over THOR.
Abstract¶
Secure Transformer inference can be made non-interactive under fully homomorphic encryption (FHE), but the computation is extremely expensive. Existing FHE-based systems reduce that cost by optimizing individual stages in isolation, each within its own component rather than against the CKKS level budget they all share. A local gain therefore may not translate into an end-to-end speedup. We present THEMIS, a co-designed system for encrypted Transformer inference organized around two design principles, one bounding the depth each kind of stage may spend and one fixing where on the modulus chain it runs. THEMIS generalizes coefficient-encoded plaintext-ciphertext matrix multiplication (PCMM) to slot-encoded ciphertexts without format conversion, retaining the same accelerated coefficientside backend and amortizing its nearly fixed cost through multi-batch packing. Two attention ciphertext-ciphertext matrix multiplication (CCMM) kernels combine the idle imaginary lane of each slot with baby-step giant-step accumulation and hoisting, so the attention multiplications issue far fewer key switches. For softmax, THEMIS conditions the first denominator with a calibrated row-wise offset and reserves high precision for the final normalization alone, which cuts the depth the reciprocal path consumes while keeping the precision of the returned probabilities. For a single stage, THEMIS accelerates PCMM and the attention CCMMs by 21.66× and up to 7.39× over MOAI (ICLR’26), and by up to 23.02× and 16.49× over Euston (S&P’26), while THEMIS’s softmax runs 2.57× faster than THOR (CCS’25) at higher precision. For the complete pipeline, THEMIS serves BERT-base on a CPU in 400.06 amortized seconds per input, with accuracy on GLUE tasks comparable to plaintext. Of that latency, the stages around bootstrapping account for only 10.83%, which is the lowest share among all works compared here.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Level-budget co-design with two principles: per-stage depth bound and fixed position on the modulus chain | llm | 0.90 |
| contribution | Slot-encoded PCMM without format conversion using the coefficient-side dense linear-algebra backend and multi-batch packing | llm | 0.90 |
| contribution | COLPAIR and DIAGPAIR attention CCMM kernels combining imaginary-lane packing, BSGS accumulation and hoisting | llm | 0.90 |
| contribution | Denominator-conditioned softmax with calibrated row-wise offset that reserves high precision for the final normalization | llm | 0.90 |
| contribution | End-to-end BERT-base pipeline where bootstrapping-adjacent stages are only 10.83% of latency | llm | 0.90 |
| schemes | CKKS | llm | 0.90 |
| libraries | OpenFHE | llm | 0.90 |
| models | BERT-base | llm | 0.90 |
| datasets | RTE | llm | 0.90 |
| datasets | SST-2 | llm | 0.90 |
| datasets | QNLI | llm | 0.90 |
| tasks | private transformer inference | llm | 0.90 |
| tasks | text classification (GLUE) | llm | 0.90 |
| hardware | Intel Xeon Platinum 8473C CPU (52 cores, 104 threads) | llm | 0.90 |
| threat model | client-server; server learns nothing about the input beyond the final output | llm | 0.90 |
| interaction | non-interactive FHE | llm | 0.90 |
| limitations | Bootstrapping is treated as a black box with fixed per-call cost and remains 89.2% of end-to-end latency | llm | 0.90 |
| limitations | Evaluation is CPU-only; GPU mapping of BTS and the PCMM backend is left as future work | llm | 0.90 |
| llm relevance | high | llm | 0.90 |
| key result | end-to-end latency: 400.06 s amortized per input (BERT-base, L = 128, batch 256, CPU, OpenFHE) | llm | 0.90 |
| key result | PCMM speed-up: 21.66x over MOAI (up to 23.02x over Euston) (Q projection R128x768 x R768x768) | llm | 0.90 |
| key result | attention CCMM speed-up: 7.39x (QK^T) and 2.18x (SV) over MOAI; up to 16.49x over Euston (12-head attention, 96 threads) | llm | 0.90 |
| key result | softmax cost: 151.59 s total / 0.59 s amortized, 13.25-bit precision vs THOR 389.38 s / 1.52 s, 8.84 bits (2.57x faster) (RTE, 1 bootstrapping vs 3) | llm | 0.90 |
| key result | share of latency around bootstrapping: 10.83% (end-to-end BERT-base pipeline) | llm | 0.90 |
| key result | bootstrapping share of end-to-end latency: 89.2% (after reducing BTS calls) | llm | 0.90 |
System card: THEMIS¶
| Group | 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK) |
| Family | CKKS framework |
| Arithmetic | Approximate real (CKKS) |
| Interaction | Non-interactive FHE |
| Model scale | Transformer (see paper) |
| Status | Active SOTA |
| Relation to the SoK | Plans all stages against one shared CKKS level budget. |
| Note | Bounds depth per stage type and fixes where on the modulus chain each runs. Generalizes coefficient-encoded PCMM. |
| Systems page | THEMIS |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 28 [p.1], MPC 6 [p.1], Secret sharing 3 [p.3], Function secret sharing 2 [p.3]
- library: OpenFHE 6 [p.2], HElib 1 [p.15], nGraph-HE 1 [p.14]
- model: Transformer 28 [p.1], BERT 11 [p.1], GPT 2 [p.3]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 18 [p.11], SST-2 8 [p.11], GLUE 2 [p.1]
- hardware: Intel Xeon 1 [p.11], GPU 1 [p.13]
- technique: Softmax 68 [p.1], Rotations 53 [p.2], GELU 12 [p.1], Chebyshev 10 [p.5], LayerNorm 10 [p.1], RNS 10 [p.1], Matrix multiplication 10 [p.2], Bootstrapping 9 [p.1], Non-interactive 7 [p.1], Relinearization 6 [p.5], Key switching 4 [p.4], Rescaling 4 [p.3], Two-party computation 4 [p.3], Polynomial approximation 3 [p.12]
- assumption: RLWE 3 [p.4], LWE 2 [p.14]
- topic: Homomorphic encryption 21 [p.1], Benchmark 6 [p.2], Private / secure inference 3 [p.13]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/1-121_themis-a-co-designed-system-for-encrypted-transformer-infere.pdf |
catalogue | 22 | 775.1 KB | df2b90c64a3f |
Database key: themis-a-co-designed-system-for-encrypted-transformer-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set themis-a-co-designed-system-for-encrypted-transformer-inference COLUMN VALUE