ENSI: Efficient non-interactive secure inference for large language models¶
Authors: Zhiyu He; Maojiang Wang; Xinwen Gao; Yuchuan Luo; Lin Liu; Shaojing Fu
Year: 2025
Venue: arXiv
type: system collection: catalogue, sok-references status: auto
Links: arXiv:2509.09424
Summary¶
Auto-extracted summary (unreviewed)
ENSI is a non-interactive CKKS-based secure inference framework for LLMs that co-designs the cryptography with the model: it pairs an optimized encoding with the lightweight BitNet (ternary-weight, LLaMA-style) architecture to simplify encrypted matrix multiplication, replaces softmax with a retraining-free sigmoid attention, and embeds bootstrapping inside RMSNorm to cut bootstrapping frequency. Compared with NEXUS on CPU it reports about 8x faster matrix multiplication (8.1x at input scale 2^6, 5.8x at 2^11) and 2.6x faster softmax, with bootstrapping reduced to about 1% of the cost.
Abstract¶
Secure inference enables privacy-preserving machine learning by leveraging cryptographic protocols that support computations on sensitive user data without exposing it. However, integrating cryptographic protocols with large language models (LLMs) presents significant challenges, as the inherent complexity of these protocols, together with LLMs' massive parameter scale and sophisticated architectures, severely limits practical usability. In this work, we propose ENSI, a novel non-interactive secure inference framework for LLMs, based on the principle of co-designing the cryptographic protocols and LLM architecture. ENSI employs an optimized encoding strategy that seamlessly integrates CKKS scheme with a lightweight LLM variant, BitNet, significantly reducing the computational complexity of encrypted matrix multiplications. In response to the prohibitive computational demands of softmax under homomorphic encryption (HE), we pioneer the integration of the sigmoid attention mechanism with HE as a seamless, retraining-free alternative. Furthermore, by embedding the Bootstrapping operation within the RMSNorm process, we efficiently refresh ciphertexts while markedly decreasing the frequency of costly bootstrapping invocations. Experimental evaluations demonstrate that ENSI achieves approximately an 8x acceleration in matrix multiplications and a 2.6x speedup in softmax inference on CPU compared to state-of-the-art method, with the proportion of bootstrapping is reduced to just 1%.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Encoding strategy integrating CKKS with BitNet to reduce encrypted matmul complexity | llm | 0.85 |
| contribution | Sigmoid attention under HE as a retraining-free softmax replacement | llm | 0.85 |
| contribution | Bootstrapping embedded in RMSNorm, reducing bootstrapping complexity from O(d) to O(1) | llm | 0.85 |
| contribution | Open-source implementation on OpenFHE, SEAL and Phantom (GPU) | llm | 0.85 |
| schemes | CKKS | llm | 0.85 |
| libraries | OpenFHE | llm | 0.85 |
| libraries | SEAL | llm | 0.85 |
| libraries | Phantom | llm | 0.85 |
| models | BitNet (LLaMA-style) | llm | 0.85 |
| tasks | private LLM inference | llm | 0.85 |
| hardware | Intel Core i9-14900K CPU (32 cores) | llm | 0.85 |
| hardware | NVIDIA A100 80GB GPU | llm | 0.85 |
| security level | 128-bit | llm | 0.85 |
| threat model | client-server, non-interactive (server computes on ciphertexts) | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| code url | https://github.com/sugarhh/ENSI (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/Microsoft/SEAL (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/sugarhh/ENSI | llm | 0.85 |
| limitations | Experiments run single-core because OpenFHE multi-threading is still under development | llm | 0.85 |
| limitations | Evaluation covers server-side runtime only; no LAN/WAN network simulation | llm | 0.85 |
| llm relevance | high | llm | 0.85 |
| key result | PCMM speed-up vs NEXUS: 8.1x (1.41 s) (input scale 2^6, CPU single core, SEAL backend) | llm | 0.85 |
| key result | PCMM speed-up vs NEXUS: 5.8x (input scale 2^11, CPU single core) | llm | 0.85 |
| key result | softmax speed-up: 2.6x (CPU vs state-of-the-art) | llm | 0.85 |
| key result | bootstrapping share of cost: ~1% (bootstrapping embedded in RMSNorm) | llm | 0.85 |
System card: ENSI¶
| Group | 0. CKKS LLM frameworks surveyed in the SoK |
| Family | CKKS · Model-modifying (N-MOD) |
| Arithmetic | Approximate real (CKKS) |
| Interaction | non-interactive FHE |
| Model scale | Llama-3-700M (BitNet b1.58) |
| Library | OpenFHE + Phantom |
| Hardware | A100 GPU |
| Packing | L-ALG (ternary weights) |
| Latency | 958 s TTFT (batch 32, 2048 tokens) |
| Open source | yes |
| Status | Active SOTA |
| Relation to the SoK | BitNet ternary weights replace multiplications with additions; sigmoid attention. |
| Note | Model-modifying; demonstrates 1.58-bit LLMs under FHE. |
| Systems page | ENSI |
SoK reference [75]¶
Z. He, M. Wang, X. Gao, Y. Luo, L. Liu, and S. Fu, “Ensi: Efficient non-interactive secure inference for large language models,” in 2025 44th International Symposium on Reliable Distributed Systems (SRDS), 2025, pp. 83–94.
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 11 [p.1], MPC 10 [p.2], TEE / confidential computing 3 [p.2]
- library: Microsoft SEAL 4 [p.9], OpenFHE 3 [p.8], Phantom 2 [p.8]
- model: LLaMA 11 [p.1], CNN 3 [p.1], Transformer 2 [p.2], GPT 1 [p.1], AlexNet 1 [p.11]
- hardware: GPU 13 [p.2], NVIDIA A100 2 [p.9]
- technique: Matrix multiplication 29 [p.1], Bootstrapping 28 [p.1], Softmax 23 [p.1], Sigmoid 13 [p.1], Attention 9 [p.1], Non-interactive 9 [p.1], Rotations 7 [p.3], Chebyshev 6 [p.7], RNS 6 [p.2], Polynomial approximation 3 [p.7], SIMD packing 3 [p.3], Fine-tuning 3 [p.2], Paterson-Stockmeyer 2 [p.7], Quantization 2 [p.4], Two-party computation 2 [p.11], Semi-honest model 2 [p.4]
- task: Genomics 2 [p.12]
- topic: Homomorphic encryption 41 [p.1], Large language models 38 [p.1], Private / secure inference 26 [p.1], Open source 6 [p.2], Benchmark 5 [p.9]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/0-017_ensi-efficient-non-interactive-secure-inference-for-large-la.pdf |
catalogue | 12 | 3.1 MB | f8235767a6cb |
|
references/sok-references/075_ensi-efficient-non-interactive-secure-inference-for-large-la.pdf |
sok-references | 12 | 3.1 MB | f8235767a6cb |
byte-identical to references/catalogue/0-017_ensi-efficient-non-interactive-secure-inference-for-large-la.pdf |
Database key: ensi-efficient-non-interactive-secure-inference-for-large-language · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set ensi-efficient-non-interactive-secure-inference-for-large-language COLUMN VALUE