TRICYCLE: Private transformer inference with tricyclic encodings¶
Authors: L. Lim; V. Kalagi; D. Agrawal; A. El Abbadi
Year: 2025
Venue: IACR ePrint 2025/1200
type: system collection: catalogue, sok-references status: auto
Links: ePrint 2025/1200
Summary¶
Auto-extracted summary (unreviewed)
Tricycle is a CKKS-based system for private transformer inference whose core is the tricyclic encoding, a packing of three-dimensional tensors into ciphertext slots that supports batched matrix multiplication across attention heads with multiplicative depth 1 and free transposes. The encoding is combined with baby-step giant-step, optimized block matrix multiplication, lazy relinearization and complexification, and a statistical max estimation method stabilizes softmax under HE. Implemented on a GPU-accelerated CKKS pipeline, Tricycle runs BERT-Base with 128 tokens in 100.5 s on a single GPU, 6x faster than Thor and 3.4x faster than Powerformer.
Abstract¶
The growing deployment of large language models (LLMs) in privacy-sensitive settings demands inference mechanisms that preserve data confidentiality. Homomorphic encryption (HE) offers a principled solution by enabling computation directly on encrypted data; however, existing solutions struggle to scale to full LLMs. We present Tricycle, a system for efficient private transformer inference. At its core, Tricycle introduces tricyclic encodings, a novel packing scheme that enables batch matrix multiplications with optimal multiplicative depth while naturally supporting multi-head attention. We demonstrate that Tricycle’s matrix multiplications are particularly amenable for integration with other optimizations including Baby-Step Giant-Step optimizations, optimized block matrix multiplications, lazy relinearization, and complexification, which significantly improve performance. We further introduce statistical max estimation, a lightweight method for stabilizing softmax under HE. We implement Tricycle end-to-end on a GPU-accelerated CKKS pipeline and evaluate it on BERT models. For BERT-Base with 128 tokens, Tricycle achieves 100.5 seconds latency on a single GPU, yielding 6× and 3.4× speedups over prior state-of-the-art systems, Thor and Powerformer, respectively. These results demonstrate that careful design of packing, algorithms, and optimizations can significantly reduce the cost of private LLM inference, bringing practical deployment closer to reality.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | tricyclic encoding of 3D tensors enabling batch matrix multiplication with optimal multiplicative depth and free transposes | llm | 0.85 |
| contribution | batch and broadcasted batch matrix multiplication algorithms for multi-head attention plus improved ciphertext-plaintext multiplication | llm | 0.85 |
| contribution | integration with BSGS, optimized block matrix multiplication, lazy relinearization and complexification | llm | 0.85 |
| contribution | statistical max estimation for stabilizing softmax under HE | llm | 0.85 |
| contribution | end-to-end GPU-accelerated CKKS implementation and ablation study on BERT models | llm | 0.85 |
| schemes | CKKS | llm | 0.85 |
| models | BERT-base | llm | 0.85 |
| tasks | private transformer inference | llm | 0.85 |
| tasks | private LLM inference | llm | 0.85 |
| hardware | single GPU (model not stated in excerpt) | llm | 0.85 |
| security level | 128-bit | llm | 0.85 |
| threat model | client-server: user encrypts the query, model owner evaluates the model and keeps the weights | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| code url | https://bitbucket.org/tricyclepaper/tricyclic-matmul (URL found in the full text) | regex | 0.50 |
| llm relevance | high | llm | 0.85 |
| key result | end-to-end inference latency: 100.5 s (BERT-Base, 128 tokens, single GPU, CKKS) | llm | 0.85 |
| key result | speed-up over Thor: 6x (BERT-Base, 128 tokens) | llm | 0.85 |
| key result | speed-up over Powerformer: 3.4x (BERT-Base, 128 tokens) | llm | 0.85 |
System card: TRICYCLE¶
| Group | 0. CKKS LLM frameworks surveyed in the SoK |
| Family | CKKS · Model-modifying (N-MOD) |
| Arithmetic | Approximate real (CKKS) |
| Interaction | non-interactive FHE |
| Model scale | BERT-Tiny |
| Library | OpenFHE |
| Hardware | 128-core CPU |
| Packing | L-CYC |
| Latency | 322 s unamortized (27 tokens) |
| Open source | no |
| Status | Active SOTA |
| Relation to the SoK | Tricyclic encodings process multiple attention heads per ciphertext; statistical max for Softmax. |
| Note | CPU-only; closed source. |
| Systems page | TRICYCLE |
SoK reference [73]¶
L. Lim, V. Kalagi, D. Agrawal, and A. El Abbadi, “Tricycle: Private transformer inference with tricyclic encodings,” Cryptology ePrint Archive, Report 2025/1200, 2025. [Online]. Available: https://eprint.iacr.org/2025/1200
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 10 [p.1], MPC 2 [p.3]
- library: HEaaN (library) 18 [p.9], Liberate.FHE 6 [p.12], OpenFHE 3 [p.9], HElib 1 [p.14]
- model: BERT 51 [p.1], Transformer 26 [p.1], LLaMA 8 [p.13], GPT 1 [p.14]
- dataset: SST-2 3 [p.12]
- hardware: GPU 20 [p.1], NVIDIA A100 1 [p.12]
- technique: Matrix multiplication 131 [p.1], Rotations 72 [p.2], Softmax 53 [p.1], Bootstrapping 23 [p.2], Relinearization 21 [p.1], LayerNorm 21 [p.1], Key switching 16 [p.2], Attention 14 [p.3], GELU 10 [p.3], Non-interactive 9 [p.3], SIMD packing 5 [p.2], Chebyshev 4 [p.11], Fine-tuning 4 [p.4], Two-party computation 3 [p.14], Rescaling 2 [p.9]
- task: Medical / healthcare 3 [p.1]
- topic: Homomorphic encryption 29 [p.1], Large language models 28 [p.1], Private / secure inference 12 [p.1], Benchmark 2 [p.3]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/0-009_tricycle-private-transformer-inference-with-tricyclic-encodi.pdf |
catalogue | 22 | 1.7 MB | 3daffd04b39d |
|
references/sok-references/073_tricycle-private-transformer-inference-with-tricyclic-encodi.pdf |
sok-references | 22 | 1.7 MB | 3daffd04b39d |
byte-identical to references/catalogue/0-009_tricycle-private-transformer-inference-with-tricyclic-encodi.pdf |
Database key: tricycle-private-transformer-inference-with-tricyclic-encodings · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set tricycle-private-transformer-inference-with-tricyclic-encodings COLUMN VALUE