Skip to content

HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation

Authors: Zirilli, Marincione, Kornaropoulos, Ateniese, Rodolà
Year: 2026
Venue: IACR ePrint 2026/1862

type: system collection: catalogue status: auto

Links: ePrint 2026/1862

Summary

Auto-extracted summary (unreviewed)

HEAT (Homomorphic Encryption-Aware Training) is a fine-tuning method for FHE language-model inference that makes the per-nonlinearity iteration counts of the polynomial/iterative approximations learnable, so the iteration counts and model weights co-adapt to the task objective without architectural changes. The authors relate iteration count to quantization bit width and bound the gap to quantization-aware training. On encrypted GPT-2 decoding under RNS-CKKS at 128-bit security on a FIDESlib GPU backend, HEAT cuts iterations by 3.1x, bootstraps by 1.6x and end-to-end latency by 1.4x (37.6 vs 54.3 s/token) while improving decode agreement with the plaintext model (top-1 82.9% vs 70.3%) and eliminating collapsed sequences.

Abstract

Fully homomorphic encryption (FHE) allows a server to run a language model directly on encrypted user prompts, but current approaches remain prohibitively slow. Ciphertexts natively support only addition, multiplication, and rotation, and multiplications may be composed only to a bounded depth before a costly bootstrapping operation is required to continue. Every nonlinearity must therefore be approximated by an iterative method; each iteration increasing the number of multiplications. A higher iteration count buys precision but exhausts the available depth more frequently and thus triggers more bootstraps, which dominate latency. We introduce Homomorphic Encryption-Aware Training (HEAT), a fine-tuning method that makes the per-nonlinearity iteration counts learnable, enabling them and the model weights to co-adapt during training. HEAT optimizes iterations with respect to the task objective, allowing the model to adapt to approximation errors encountered during inference without architectural changes or retraining from scratch. We further relate iteration count to quantization bit width and bound, at fixed weights, the gap between our objective and quantization-aware training. On encrypted GPT-2 decoding, HEAT reduces iterations by 3.1×, bootstraps by 1.6×, and end-to-end latency by 1.4×, while improving decode agreement over the calibrated encrypted baseline. https://github.com/gladia-research-group/heat
Source: pdf

Extracted details

Field Value Source Confidence
contribution HEAT: fine-tuning with learnable per-nonlinearity approximation iteration counts co-adapted with weights llm 0.85
contribution theoretical link between iteration count and quantization bit width, with a bound relative to quantization-aware training llm 0.85
contribution end-to-end encrypted GPT-2 decoding evaluation on a CKKS GPU backend llm 0.85
contribution analysis showing Softmax refinement as the main source of encrypted drift llm 0.85
contribution open-source code, Lean 4 proofs and released model weights llm 0.85
schemes CKKS llm 0.85
libraries FIDESlib llm 0.85
models GPT-2 llm 0.85
datasets C4 llm 0.85
datasets HellaSwag llm 0.85
datasets PIQA llm 0.85
datasets OpenWebText llm 0.85
tasks private LLM inference (encrypted autoregressive decoding) llm 0.85
tasks language modeling llm 0.85
hardware GPU (via FIDESlib CKKS backend; model not named in excerpt) llm 0.85
security level 128-bit llm 0.85
threat model non-interactive client-server; server evaluates the model on the encrypted prompt without mutual trust llm 0.85
interaction non-interactive FHE llm 0.85
code url https://github.com/gladia-research-group/heat (URL found in the full text) regex 0.50
code url https://huggingface.co/gladia/heat-gpt2-small-openwebtext (URL found in the full text) regex 0.50
code url https://github.com/gladia-research-group/heat llm 0.85
llm relevance high llm 0.85
key result approximation iterations: 3.1x fewer (encrypted GPT-2 decoding vs calibrated baseline) llm 0.85
key result bootstraps: 1.6x fewer (encrypted GPT-2 decoding vs calibrated baseline) llm 0.85
key result end-to-end latency: 37.6 s/token vs 54.3 s/token (1.4x faster) (RNS-CKKS, 128-bit security, FIDESlib backend) llm 0.85
key result decode top-1 agreement with plaintext: 82.9% vs 70.3% (calibrated baseline); collapsed sequences 0/64 vs 7/64 (encrypted GPT-2 decoding) llm 0.85
key result C4 perplexity: 28.7 (HEAT) vs 27.9 (plaintext GPT-2) (plaintext evaluation of HEAT-tuned model) llm 0.85
key result HellaSwag / PIQA accuracy: 28.7% / 62.8% (HEAT) vs 28.9% / 62.9% (GPT-2) (plaintext evaluation) llm 0.85

System card: HEAT

Group 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK)
Family CKKS + HE-aware training
Arithmetic Approximate real (CKKS)
Interaction Non-interactive FHE
Model scale Language model (see paper)
Status Active SOTA
Relation to the SoK Fine-tunes weights to tolerate cheaper approximations.
Note Model-modifying (N-MOD style): trades a short training step for fewer bootstraps.
Systems page HEAT

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 17 [p.3], MPC 3 [p.12]
  • library: FIDESlib 5 [p.6], OpenFHE 3 [p.10], Orion 1 [p.11], HuggingFace 1 [p.3]
  • model: GPT 39 [p.1], ViT 5 [p.24], Transformer 3 [p.12], CNN 2 [p.10]
  • dataset: GLUE tasks (QNLI/RTE/MRPC/CoLA/QQP/MNLI/STS-B) 6 [p.25], GLUE 2 [p.25], HellaSwag 2 [p.25], SST-2 1 [p.25]
  • hardware: NVIDIA A100 7 [p.6], GPU 7 [p.6], Intel Xeon 2 [p.6], CUDA 2 [p.10]
  • technique: Quantization 42 [p.1], Softmax 35 [p.4], Fine-tuning 30 [p.1], LayerNorm 24 [p.4], Goldschmidt 21 [p.3], GELU 15 [p.7], Bootstrapping 14 [p.1], Rotations 6 [p.1], KV cache 5 [p.3], Non-interactive 5 [p.1], ReLU 3 [p.9], RNS 3 [p.11], Two-party computation 3 [p.12], Rescaling 2 [p.19], Chebyshev 2 [p.3], Remez 2 [p.3], Newton iteration 2 [p.3]
  • assumption: 128-bit security 2 [p.6]
  • task: Language modeling / generation 9 [p.1]
  • topic: Homomorphic encryption 45 [p.1], Private / secure inference 17 [p.4], Large language models 6 [p.10], Open source 4 [p.1], Benchmark 2 [p.25]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/1-127_heat-faster-fully-homomorphic-inference-via-approximations-w.pdf catalogue 29 720.8 KB df407673bd56

Database key: heat-faster-fully-homomorphic-inference-via-approximations-weights-co · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set heat-faster-fully-homomorphic-inference-via-approximations-weights-co COLUMN VALUE