Skip to content

ENSI: Efficient non-interactive secure inference for large language models

Authors: Zhiyu He; Maojiang Wang; Xinwen Gao; Yuchuan Luo; Lin Liu; Shaojing Fu
Year: 2025
Venue: arXiv

type: system collection: catalogue, sok-references status: auto

Links: arXiv:2509.09424

Summary

Auto-extracted summary (unreviewed)

ENSI is a non-interactive CKKS-based secure inference framework for LLMs that co-designs the cryptography with the model: it pairs an optimized encoding with the lightweight BitNet (ternary-weight, LLaMA-style) architecture to simplify encrypted matrix multiplication, replaces softmax with a retraining-free sigmoid attention, and embeds bootstrapping inside RMSNorm to cut bootstrapping frequency. Compared with NEXUS on CPU it reports about 8x faster matrix multiplication (8.1x at input scale 2^6, 5.8x at 2^11) and 2.6x faster softmax, with bootstrapping reduced to about 1% of the cost.

Abstract

Secure inference enables privacy-preserving machine learning by leveraging cryptographic protocols that support computations on sensitive user data without exposing it. However, integrating cryptographic protocols with large language models (LLMs) presents significant challenges, as the inherent complexity of these protocols, together with LLMs' massive parameter scale and sophisticated architectures, severely limits practical usability. In this work, we propose ENSI, a novel non-interactive secure inference framework for LLMs, based on the principle of co-designing the cryptographic protocols and LLM architecture. ENSI employs an optimized encoding strategy that seamlessly integrates CKKS scheme with a lightweight LLM variant, BitNet, significantly reducing the computational complexity of encrypted matrix multiplications. In response to the prohibitive computational demands of softmax under homomorphic encryption (HE), we pioneer the integration of the sigmoid attention mechanism with HE as a seamless, retraining-free alternative. Furthermore, by embedding the Bootstrapping operation within the RMSNorm process, we efficiently refresh ciphertexts while markedly decreasing the frequency of costly bootstrapping invocations. Experimental evaluations demonstrate that ENSI achieves approximately an 8x acceleration in matrix multiplications and a 2.6x speedup in softmax inference on CPU compared to state-of-the-art method, with the proportion of bootstrapping is reduced to just 1%.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution Encoding strategy integrating CKKS with BitNet to reduce encrypted matmul complexity llm 0.85
contribution Sigmoid attention under HE as a retraining-free softmax replacement llm 0.85
contribution Bootstrapping embedded in RMSNorm, reducing bootstrapping complexity from O(d) to O(1) llm 0.85
contribution Open-source implementation on OpenFHE, SEAL and Phantom (GPU) llm 0.85
schemes CKKS llm 0.85
libraries OpenFHE llm 0.85
libraries SEAL llm 0.85
libraries Phantom llm 0.85
models BitNet (LLaMA-style) llm 0.85
tasks private LLM inference llm 0.85
hardware Intel Core i9-14900K CPU (32 cores) llm 0.85
hardware NVIDIA A100 80GB GPU llm 0.85
security level 128-bit llm 0.85
threat model client-server, non-interactive (server computes on ciphertexts) llm 0.85
interaction non-interactive FHE llm 0.85
code url https://github.com/sugarhh/ENSI (URL found in the full text) regex 0.50
code url https://github.com/Microsoft/SEAL (URL found in the full text) regex 0.50
code url https://github.com/sugarhh/ENSI llm 0.85
limitations Experiments run single-core because OpenFHE multi-threading is still under development llm 0.85
limitations Evaluation covers server-side runtime only; no LAN/WAN network simulation llm 0.85
llm relevance high llm 0.85
key result PCMM speed-up vs NEXUS: 8.1x (1.41 s) (input scale 2^6, CPU single core, SEAL backend) llm 0.85
key result PCMM speed-up vs NEXUS: 5.8x (input scale 2^11, CPU single core) llm 0.85
key result softmax speed-up: 2.6x (CPU vs state-of-the-art) llm 0.85
key result bootstrapping share of cost: ~1% (bootstrapping embedded in RMSNorm) llm 0.85

System card: ENSI

Group 0. CKKS LLM frameworks surveyed in the SoK
Family CKKS · Model-modifying (N-MOD)
Arithmetic Approximate real (CKKS)
Interaction non-interactive FHE
Model scale Llama-3-700M (BitNet b1.58)
Library OpenFHE + Phantom
Hardware A100 GPU
Packing L-ALG (ternary weights)
Latency 958 s TTFT (batch 32, 2048 tokens)
Open source yes
Status Active SOTA
Relation to the SoK BitNet ternary weights replace multiplications with additions; sigmoid attention.
Note Model-modifying; demonstrates 1.58-bit LLMs under FHE.
Systems page ENSI

SoK reference [75]

Z. He, M. Wang, X. Gao, Y. Luo, L. Liu, and S. Fu, “Ensi: Efficient non-interactive secure inference for large language models,” in 2025 44th International Symposium on Reliable Distributed Systems (SRDS), 2025, pp. 83–94.

Listed on the reference page of the site.

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 11 [p.1], MPC 10 [p.2], TEE / confidential computing 3 [p.2]
  • library: Microsoft SEAL 4 [p.9], OpenFHE 3 [p.8], Phantom 2 [p.8]
  • model: LLaMA 11 [p.1], CNN 3 [p.1], Transformer 2 [p.2], GPT 1 [p.1], AlexNet 1 [p.11]
  • hardware: GPU 13 [p.2], NVIDIA A100 2 [p.9]
  • technique: Matrix multiplication 29 [p.1], Bootstrapping 28 [p.1], Softmax 23 [p.1], Sigmoid 13 [p.1], Attention 9 [p.1], Non-interactive 9 [p.1], Rotations 7 [p.3], Chebyshev 6 [p.7], RNS 6 [p.2], Polynomial approximation 3 [p.7], SIMD packing 3 [p.3], Fine-tuning 3 [p.2], Paterson-Stockmeyer 2 [p.7], Quantization 2 [p.4], Two-party computation 2 [p.11], Semi-honest model 2 [p.4]
  • task: Genomics 2 [p.12]
  • topic: Homomorphic encryption 41 [p.1], Large language models 38 [p.1], Private / secure inference 26 [p.1], Open source 6 [p.2], Benchmark 5 [p.9]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/0-017_ensi-efficient-non-interactive-secure-inference-for-large-la.pdf catalogue 12 3.1 MB f8235767a6cb
references/sok-references/075_ensi-efficient-non-interactive-secure-inference-for-large-la.pdf sok-references 12 3.1 MB f8235767a6cb byte-identical to references/catalogue/0-017_ensi-efficient-non-interactive-secure-inference-for-large-la.pdf

Database key: ensi-efficient-non-interactive-secure-inference-for-large-language · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set ensi-efficient-non-interactive-secure-inference-for-large-language COLUMN VALUE