Systems catalogue¶
Papers behind the systems table (docs/systems.html): LLM frameworks, libraries, compilers, accelerators and alternatives. Files are named group-id_slug.
Folder: papers/references/catalogue/
0. CKKS LLM frameworks surveyed in the SoK¶
| System | Paper | Year | Library | Hardware | Latency | Code |
|---|---|---|---|---|---|---|
| POLARIS | SoK: Private LLM Inference using Approximate Homomorphic Encryption | 2026 | OpenFHE + FIDESlib | A100 GPU | 18 s (BERT-Tiny), 83 s (BERT-Mini), unamortized | |
| THOR | THOR: Secure transformer inference with homomorphic encryption | 2025 | Liberate.FHE | A100 GPU | 602 s / 128-token sequence | code |
| CEP / EncryptedLLM (GPU-accelerated FHE GPT-2) | EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption | 2024 | Custom OpenFHE GPU backend | A100 GPU | ~100 s per generated token | |
| CACHEMIR | Cachemir: Fully Homomorphic Encrypted Inference of Generative Large Language Model with KV Cache | 2026 | Lattigo + Phantom | A100 GPU | 97 s time-per-output-token | |
| CERIUM | A scalable multi-GPU framework for encrypted large-model inference | 2025 | Custom multi-GPU CKKS compiler | 1-8x A100/H100/B200 GPUs | 134 s prefill (128 tokens, 8x B200); 66 s BERT-Base on A100 | |
| SplitPrompt | Scaling up FHE-based Privacy-Preserving ML: Higher Throughput, Longer Inputs for LLama-3-8B (v1: Scaling up Privacy-Preserving ML: A CKKS Implementation of Llama-2-7B) | 2026 | HEaaN2 | 8x RTX 4090 GPUs | 33-34 s per output token; 85 s summarization with 4096 tokens (128 encrypted) | |
| ARION | ARION: Attention-optimized transformer inference on encrypted data | 2025 | Lattigo | 64-core CPU | 9.3 s (Tiny) / 225 s (Base) amortized per sequence, batch 256 | |
| ELLMO | ELLMo: Packing- and Depth-Aware Encrypted Transformer Inference | 2026 | OpenFHE + FIDESlib | H200 GPU | 8 s unamortized | |
| RL (Rovida & Leporati) | Transformer-based language models and homomorphic encryption: An intersection with bert-tiny | 2024 | OpenFHE | Apple M1 Pro CPU | ~672 s unamortized (32 tokens) | code |
| TRICYCLE | TRICYCLE: Private transformer inference with tricyclic encodings | 2025 | OpenFHE | 128-core CPU | 322 s unamortized (27 tokens) | |
| EFLA (Encryption-Friendly LLM Architecture) | Encryption-friendly LLM architecture | 2025 | HEaaN | RTX 4090 GPU | 27 s per layer | code |
| PRITRAN | Pritran: Privacy-preserving inference for transformer-based language models under fully homomorphic encryption | 2026 | OpenFHE | 64-core CPU | 398 s (Tiny) / ~2000 s (Mini) unamortized | |
| EUSTON | Euston: Efficient and User-Friendly Secure Transformer Inference with Non-Interactivity | 2026 | SEAL + Phantom | RTX 6000 GPU | 46 s (BERT-Base) / 136 s (GPT-2-1.5B) / 484 s (Llama-3-8B) amortized, batch 32 | |
| Powerformer | Powerformer: Efficient and high-accuracy privacy-preserving language model with homomorphic encryption | 2025 | Liberate.FHE | A100 GPU | 345 s unamortized (128 tokens) | |
| MOAI | MOAI: Module-optimizing architecture for non-interactive secure transformer inference | 2026 | SEAL + Phantom | H200 / A100 GPU | 141 s BERT-Base amortized; 225 s Llama-3-8B decode (8 tokens) | |
| NEXUS | Secure transformer inference made non-interactive | 2025 | SEAL + Phantom | 4x A100 GPUs | 37 s BERT-Base amortized (batch 32); 52 s/token Llama-3-8B | code |
| STIP | STIP: Efficient and secure non-interactive transformer inference via compact packing | 2026 | SEAL + Phantom | A100 GPU | 64 s (BERT-Base) / 142 s (GPT-2-1.5B) / 407 s (Llama-3-8B) amortized | |
| ENSI | ENSI: Efficient non-interactive secure inference for large language models | 2025 | OpenFHE + Phantom | A100 GPU | 958 s TTFT (batch 32, 2048 tokens) | |
| Polyformer | Converting transformers to polynomial form for secure inference over homomorphic encryption | 2024 | HELayers | A100 GPU | 305 s (128 tokens next-word prediction) | |
| SecretVoice 1.0 | Secretvoice1.0: Non-interactive secure transformer inference | 2026 | Microsoft SEAL | 1-core CPU | 1488 s per encoder block (32 tokens) | |
| RZW (encrypted LoRA split model) | Practical secure inference algorithm for a fine-tuned large language model based on fully homomorphic encryption | 2025 | (see paper) | (see paper) | 1.61 s/token (reported) |
1. Newer CKKS / FHE LLM frameworks (2026, not in the SoK)¶
2. CKKS core and variants¶
3. CKKS bootstrapping and operation advances¶
4. Alternative FHE schemes¶
5. Interactive MPC and HE+MPC hybrids¶
6. Trusted hardware, statistical and integrity-only alternatives¶
| System | Paper | Year | Library | Hardware | Latency | Code |
|---|---|---|---|---|---|---|
| NVIDIA Confidential Computing (H100/Blackwell) | NVIDIA Confidential Computing | 2023 | ||||
| Confidential computing on H100: benchmark | Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study | 2024 | ||||
| Apple Private Cloud Compute | Private Cloud Compute: A new frontier for AI privacy in the cloud | 2024 | ||||
| Intel TDX / AMD SEV-SNP | Confidential VMs (Intel TDX, AMD SEV-SNP) | 2021 | code | |||
| Slalom | Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware | 2019 | code | |||
| Split-N-Denoise (SnD) | Split-and-Denoise: Protect Large Language Model Inference with Local Differential Privacy | 2024 | ||||
| InferDPT | InferDPT: Privacy-Preserving Inference for Closed-box Large Language Model | 2023 | ||||
| PrivateLoRA | PrivateLoRA for Efficient Privacy Preserving LLM | 2023 | ||||
| zkLLM | zkLLM: Zero Knowledge Proofs for Large Language Models | 2024 | code | |||
| EZKL | EZKL: zkSNARKs for ONNX models | 2023 | code |