HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation¶
Authors: Zirilli, Marincione, Kornaropoulos, Ateniese, Rodolà
Year: 2026
Venue: IACR ePrint 2026/1862
type: system collection: catalogue status: auto
Links: ePrint 2026/1862
Summary¶
Auto-extracted summary (unreviewed)
HEAT (Homomorphic Encryption-Aware Training) is a fine-tuning method for FHE language-model inference that makes the per-nonlinearity iteration counts of the polynomial/iterative approximations learnable, so the iteration counts and model weights co-adapt to the task objective without architectural changes. The authors relate iteration count to quantization bit width and bound the gap to quantization-aware training. On encrypted GPT-2 decoding under RNS-CKKS at 128-bit security on a FIDESlib GPU backend, HEAT cuts iterations by 3.1x, bootstraps by 1.6x and end-to-end latency by 1.4x (37.6 vs 54.3 s/token) while improving decode agreement with the plaintext model (top-1 82.9% vs 70.3%) and eliminating collapsed sequences.
Abstract¶
Fully homomorphic encryption (FHE) allows a server to run a language model directly on encrypted user prompts, but current approaches remain prohibitively slow. Ciphertexts natively support only addition, multiplication, and rotation, and multiplications may be composed only to a bounded depth before a costly bootstrapping operation is required to continue. Every nonlinearity must therefore be approximated by an iterative method; each iteration increasing the number of multiplications. A higher iteration count buys precision but exhausts the available depth more frequently and thus triggers more bootstraps, which dominate latency. We introduce Homomorphic Encryption-Aware Training (HEAT), a fine-tuning method that makes the per-nonlinearity iteration counts learnable, enabling them and the model weights to co-adapt during training. HEAT optimizes iterations with respect to the task objective, allowing the model to adapt to approximation errors encountered during inference without architectural changes or retraining from scratch. We further relate iteration count to quantization bit width and bound, at fixed weights, the gap between our objective and quantization-aware training. On encrypted GPT-2 decoding, HEAT reduces iterations by 3.1×, bootstraps by 1.6×, and end-to-end latency by 1.4×, while improving decode agreement over the calibrated encrypted baseline. https://github.com/gladia-research-group/heat
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | HEAT: fine-tuning with learnable per-nonlinearity approximation iteration counts co-adapted with weights | llm | 0.85 |
| contribution | theoretical link between iteration count and quantization bit width, with a bound relative to quantization-aware training | llm | 0.85 |
| contribution | end-to-end encrypted GPT-2 decoding evaluation on a CKKS GPU backend | llm | 0.85 |
| contribution | analysis showing Softmax refinement as the main source of encrypted drift | llm | 0.85 |
| contribution | open-source code, Lean 4 proofs and released model weights | llm | 0.85 |
| schemes | CKKS | llm | 0.85 |
| libraries | FIDESlib | llm | 0.85 |
| models | GPT-2 | llm | 0.85 |
| datasets | C4 | llm | 0.85 |
| datasets | HellaSwag | llm | 0.85 |
| datasets | PIQA | llm | 0.85 |
| datasets | OpenWebText | llm | 0.85 |
| tasks | private LLM inference (encrypted autoregressive decoding) | llm | 0.85 |
| tasks | language modeling | llm | 0.85 |
| hardware | GPU (via FIDESlib CKKS backend; model not named in excerpt) | llm | 0.85 |
| security level | 128-bit | llm | 0.85 |
| threat model | non-interactive client-server; server evaluates the model on the encrypted prompt without mutual trust | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| code url | https://github.com/gladia-research-group/heat (URL found in the full text) | regex | 0.50 |
| code url | https://huggingface.co/gladia/heat-gpt2-small-openwebtext (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/gladia-research-group/heat | llm | 0.85 |
| llm relevance | high | llm | 0.85 |
| key result | approximation iterations: 3.1x fewer (encrypted GPT-2 decoding vs calibrated baseline) | llm | 0.85 |
| key result | bootstraps: 1.6x fewer (encrypted GPT-2 decoding vs calibrated baseline) | llm | 0.85 |
| key result | end-to-end latency: 37.6 s/token vs 54.3 s/token (1.4x faster) (RNS-CKKS, 128-bit security, FIDESlib backend) | llm | 0.85 |
| key result | decode top-1 agreement with plaintext: 82.9% vs 70.3% (calibrated baseline); collapsed sequences 0/64 vs 7/64 (encrypted GPT-2 decoding) | llm | 0.85 |
| key result | C4 perplexity: 28.7 (HEAT) vs 27.9 (plaintext GPT-2) (plaintext evaluation of HEAT-tuned model) | llm | 0.85 |
| key result | HellaSwag / PIQA accuracy: 28.7% / 62.8% (HEAT) vs 28.9% / 62.9% (GPT-2) (plaintext evaluation) | llm | 0.85 |
System card: HEAT¶
| Group | 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK) |
| Family | CKKS + HE-aware training |
| Arithmetic | Approximate real (CKKS) |
| Interaction | Non-interactive FHE |
| Model scale | Language model (see paper) |
| Status | Active SOTA |
| Relation to the SoK | Fine-tunes weights to tolerate cheaper approximations. |
| Note | Model-modifying (N-MOD style): trades a short training step for fewer bootstraps. |
| Systems page | HEAT |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 17 [p.3], MPC 3 [p.12]
- library: FIDESlib 5 [p.6], OpenFHE 3 [p.10], Orion 1 [p.11], HuggingFace 1 [p.3]
- model: GPT 39 [p.1], ViT 5 [p.24], Transformer 3 [p.12], CNN 2 [p.10]
- dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 6 [p.25], GLUE 2 [p.25], HellaSwag 2 [p.25], SST-2 1 [p.25]
- hardware: NVIDIA A100 7 [p.6], GPU 7 [p.6], Intel Xeon 2 [p.6], CUDA 2 [p.10]
- technique: Quantization 42 [p.1], Softmax 35 [p.4], Fine-tuning 30 [p.1], LayerNorm 24 [p.4], Goldschmidt 21 [p.3], GELU 15 [p.7], Bootstrapping 14 [p.1], Rotations 6 [p.1], KV cache 5 [p.3], Non-interactive 5 [p.1], ReLU 3 [p.9], RNS 3 [p.11], Two-party computation 3 [p.12], Rescaling 2 [p.19], Chebyshev 2 [p.3], Remez 2 [p.3], Newton iteration 2 [p.3]
- assumption: 128-bit security 2 [p.6]
- task: Language modeling / generation 9 [p.1]
- topic: Homomorphic encryption 45 [p.1], Private / secure inference 17 [p.4], Large language models 6 [p.10], Open source 4 [p.1], Benchmark 2 [p.25]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/1-127_heat-faster-fully-homomorphic-inference-via-approximations-w.pdf |
catalogue | 29 | 720.8 KB | df407673bd56 |
Database key: heat-faster-fully-homomorphic-inference-via-approximations-weights-co · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set heat-faster-fully-homomorphic-inference-via-approximations-weights-co COLUMN VALUE