Skip to content

ROSETTA: Efficient and Accurate Privacy-Preserving LLM Decoding via Hybrid CKKS/TFHE Evaluation

Authors: Yu, Zhang, Kong, Ding, Chen, Yu
Year: 2026
Venue: IACR ePrint 2026/2047

type: system collection: catalogue, schemes status: auto

Links: DOI · ePrint 2026/2047

Summary

Auto-extracted summary (unreviewed)

ROSETTA is a hybrid CKKS/TFHE framework for FHE-based private LLM decoding that targets the nonlinear operations dominating the decode stage. It contributes an adaptive segmented lookup-table protocol built on TFHE programmable bootstrapping for accurate nonlinear evaluation, and a scheme-aware operator-selection framework that assigns each nonlinear operator to CKKS or TFHE (with level allocation) via shortest-path search to minimize end-to-end latency. On GPT-2, TinyLlama-1.1B and LLaMA-3-8B it reaches up to 4.8x Softmax speedup and 1.5-2.1x end-to-end speedup over the pure-CKKS baseline CacheMir with negligible perplexity degradation.

Abstract

Generative large language models (LLMs) have achieved state-ofthe-art performance on many real-world tasks such as code generation and question answering. These models predominantly rely on an autoregressive decoding strategy that generates output tokens sequentially. However, their pervasive deployment raises serious privacy concerns, motivating private inference frameworks based on fully homomorphic encryption (FHE). A major limitation of existing FHE frameworks is their inefficiency in evaluating nonlinear operations, which incur substantial overhead and dominate the decode stage. In this paper, we propose ROSETTA, a hybrid CKKS/TFHE framework that overcomes this limitation. We first observe that nonlinear operations in the decode stage exhibit heterogeneous workload patterns, which can be handled effectively via a hybrid approach. We then realize this with two key contributions: 1) an adaptive segmented lookup-table protocol based on TFHE that enables efficient and accurate evaluation of nonlinear operations; and 2) a schemeaware operator-selection framework that automatically assigns each nonlinear operator to CKKS or TFHE to minimize end-to-end decoding latency. We demonstrate that ROSETTA achieves up to 4.8× Softmax speedup and 1.5–2.1× end-to-end speedup over the SOTA framework CacheMir.
Source: pdf

Extracted details

Field Value Source Confidence
contribution Observation that decode-stage nonlinear operations have heterogeneous workloads suited to a hybrid CKKS/TFHE approach llm 0.85
contribution Adaptive segmented lookup-table protocol on TFHE for efficient and accurate nonlinear evaluation llm 0.85
contribution Scheme-aware operator-selection framework assigning each nonlinear operator to CKKS or TFHE and allocating CKKS levels via shortest-path search llm 0.85
contribution CPU (Lattigo) and GPU (customized Phantom) implementation and comparison against CacheMir, NEXUS, MOAI and PEGASUS llm 0.85
schemes CKKS llm 0.85
schemes TFHE llm 0.85
libraries Lattigo llm 0.85
libraries Phantom llm 0.85
models GPT-2 llm 0.85
models TinyLlama-1.1B llm 0.85
models LLaMA-3-8B llm 0.85
tasks private LLM inference (autoregressive decoding) llm 0.85
tasks language modeling (perplexity) llm 0.85
hardware Intel Xeon Gold 6240R CPU (2.40 GHz, 48 threads, 384 GB RAM) llm 0.85
hardware NVIDIA A100 80 GB GPU llm 0.85
security level 128-bit llm 0.85
threat model client-server; client learns nothing beyond the output, server learns nothing about the input llm 0.85
interaction non-interactive FHE llm 0.85
code url https://github.com/tuneinsight/lattigo.EPFL-LDS (URL found in the full text) regex 0.50
llm relevance high llm 0.85
key result Softmax speedup: up to 4.8x (vs CacheMir (pure CKKS)) llm 0.85
key result end-to-end decoding speedup: 1.5-2.1x (vs CacheMir) llm 0.85
key result Softmax latency: 31-32 s (14.1-18.0 bits precision) (range [-8,8], context 512-8k, vs CacheMir 149 s) llm 0.85
key result Softmax latency: 51 s (12.7-16.6 bits) (range [-16,16], context 512-8k) llm 0.85
key result wide-range Softmax latency: 117-122 s (12-18 bits) (range [-128,128], vs Cho et al. 192-198 s) llm 0.85
key result perplexity degradation: negligible (8B models) llm 0.85

System card: ROSETTA

Group 1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK)
Family Hybrid CKKS + TFHE
Arithmetic Approximate + LUT
Interaction Non-interactive FHE
Model scale Generative LLM (see paper)
Status Active SOTA
Relation to the SoK Uses CKKS for linear layers and TFHE for decode-stage non-linearities.
Note A scheme-switching alternative to the SoK's pure polynomial-approximation approach.
Systems page ROSETTA

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 107 [p.1], TFHE 101 [p.1], Scheme switching 28 [p.2], MPC 8 [p.1]
  • library: Lattigo 4 [p.10], Phantom 2 [p.10], Orion 1 [p.14]
  • model: LLaMA 20 [p.1], Transformer 15 [p.4], GPT 11 [p.1], ViT 1 [p.15], CNN 1 [p.14], Mixture of experts 1 [p.15]
  • dataset: WikiText 5 [p.12], LAMBADA 4 [p.12]
  • hardware: GPU 5 [p.10], CUDA 2 [p.10], NVIDIA A100 1 [p.10], Intel Xeon 1 [p.10]
  • technique: Lookup table 55 [p.2], Programmable bootstrapping 30 [p.1], Softmax 25 [p.1], GELU 18 [p.9], SIMD packing 13 [p.2], Bootstrapping 11 [p.1], Rotations 11 [p.4], LayerNorm 7 [p.8], Attention 5 [p.4], Convolution 4 [p.14], Polynomial approximation 3 [p.2], Chebyshev 3 [p.13], Goldschmidt 3 [p.2], ReLU 3 [p.9], Quantization 3 [p.15], KV cache 3 [p.5], Two-party computation 3 [p.5], Matrix multiplication 2 [p.4], Fine-tuning 2 [p.11], Non-interactive 2 [p.15]
  • assumption: RLWE 40 [p.2], LWE 36 [p.4]
  • task: Language modeling / generation 15 [p.1], Medical / healthcare 3 [p.1]
  • topic: Large language models 35 [p.1], Homomorphic encryption 31 [p.1], Private / secure inference 14 [p.1], Benchmark 6 [p.11], Open source 2 [p.13]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/1-122_rosetta-efficient-and-accurate-privacy-preserving-llm-decodi.pdf catalogue 15 1.9 MB a42865cb2606
references/schemes/s40-1_eprint-2026-2047.pdf schemes 15 1.9 MB a42865cb2606 byte-identical to references/catalogue/1-122_rosetta-efficient-and-accurate-privacy-preserving-llm-decodi.pdf

Database key: rosetta-efficient-and-accurate-privacy-preserving-llm-decoding-via · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set rosetta-efficient-and-accurate-privacy-preserving-llm-decoding-via COLUMN VALUE