Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption¶
Authors: Sayan Biswas; Philippe Chartier; Akash Dhasade; Tom Jurien; David Kerriou; Anne-Marie Kerrmarec; Mohammed Lemou; Franklin Tranie; Martijn de Vos; Milos Vujasinovic
Year: 2025
Venue: arXiv preprint
type: system collection: ppml status: auto
Links: arXiv:2509.01253
Summary¶
Auto-extracted summary (unreviewed)
Safhire is a hybrid FHE inference framework in which the server evaluates linear layers under an RLWE-based encryption while the client computes exact non-linear activations in plaintext, which removes bootstrapping and activation approximations; randomized shuffling of intermediate values protects model confidentiality, and fast ciphertext packing plus partial trace extraction reduce latency. Implemented in Julia with CPU and GPU support, it achieves up to 7.6x lower latency than Orion and 7.4x lower than Concrete-ML on ResNet-20/18/34 over CIFAR-10, Tiny and ImageNet with comparable accuracy and manageable communication.
Abstract¶
In contemporary cloud-based services, protecting users' sensitive data and ensuring the confidentiality of the server's model are critical. Fully homomorphic encryption (FHE) enables inference directly on encrypted inputs, but its practicality is hindered by expensive bootstrapping and inefficient approximations of non-linear activations. We introduce Safhire, a hybrid inference framework that executes linear layers under encryption on the server while offloading non- linearities to the client in plaintext. This design eliminates bootstrapping, supports exact activations, and significantly reduces computation cost. To safeguard model confidentiality despite client access to intermediate outputs, Safhire applies randomized shuffling, which obfuscates intermediate values and makes it practically impossible for a client to reconstruct the model. To further reduce inference latency, Safhire incorporates advanced optimizations such as fast ciphertext packing and partial extraction. Evaluations on multiple standard models and datasets show that Safhire achieves up to 7.6x and 7.4x lower inference latency than Orion and Concrete-ML, respectively, which are two state-of-the-art FHE baselines, with manageable communication overhead and comparable accuracy. These results underline the practicality of our hybrid FHE inference scheme.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | Hybrid client-server FHE inference that eliminates bootstrapping by offloading non-linearities to the client | llm | 0.85 |
| contribution | Randomized shuffling of intermediate outputs to protect model confidentiality | llm | 0.85 |
| contribution | Fast ciphertext packing and partial trace extraction optimizations | llm | 0.85 |
| contribution | Multi-threaded CPU and GPU implementation in Julia | llm | 0.85 |
| schemes | RLWE-based leveled HE (custom) | llm | 0.85 |
| libraries | Julia | llm | 0.85 |
| libraries | NNlib.jl | llm | 0.85 |
| libraries | FFTW.jl | llm | 0.85 |
| libraries | Permutations.jl | llm | 0.85 |
| libraries | Orion (baseline) | llm | 0.85 |
| libraries | Concrete ML (baseline) | llm | 0.85 |
| models | ResNet-20 | llm | 0.85 |
| models | ResNet-18 | llm | 0.85 |
| models | ResNet-34 | llm | 0.85 |
| datasets | CIFAR-10 | llm | 0.85 |
| datasets | Tiny ImageNet | llm | 0.85 |
| datasets | ImageNet | llm | 0.85 |
| tasks | image classification | llm | 0.85 |
| tasks | private CNN inference | llm | 0.85 |
| hardware | 2x Intel Xeon E5-2690 v4 (56 logical cores, 925 GB RAM) | llm | 0.85 |
| hardware | 2x Intel Xeon Platinum 8272L (104 logical cores, 2975 GB RAM) | llm | 0.85 |
| hardware | NVIDIA A100-SXM4 80GB GPU with 24-core AMD EPYC 7543 | llm | 0.85 |
| security level | at least 128-bit (Albrecht LWE estimator) | llm | 0.85 |
| threat model | client-server; client sees shuffled intermediate activations; model confidentiality via shuffling | llm | 0.85 |
| interaction | interactive (HE+MPC) | llm | 0.85 |
| limitations | Client must participate in every non-linear layer, adding communication overhead | llm | 0.85 |
| limitations | Max pooling replaced by average pooling for ResNet-34 on ImageNet | llm | 0.85 |
| llm relevance | medium | llm | 0.85 |
| key result | inference latency speed-up vs Orion: up to 7.6x (1.5x to 10.5x range) (single-threaded CPU, ResNet models) | llm | 0.85 |
| key result | inference latency speed-up vs Concrete-ML: up to 7.4x (FHE baselines comparison) | llm | 0.85 |
| key result | Orion baseline latency (motivation): 17 min / 80 min / 180 min (ResNet-20 CIFAR-10 / ResNet-18 Tiny / ResNet-34 ImageNet, bootstrapping 62.7 to 82.2% of time) | llm | 0.85 |
| key result | correctness: no discrepancies vs quantized cleartext model (all tested precision and trace-extraction configurations) | llm | 0.85 |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 5 [p.9], TFHE 5 [p.2], Differential privacy 5 [p.2], Multiparty HE 3 [p.13], MPC 3 [p.13], Leveled / somewhat HE 2 [p.12]
- library: Zama 2 [p.13], Orion 2 [p.9], EVA / CHET 2 [p.12], Concrete (Zama) 1 [p.14]
- model: ResNet 45 [p.1], CNN 11 [p.1], GPT 1 [p.13]
- dataset: CIFAR-10 18 [p.1], ImageNet 13 [p.1], MNIST 1 [p.12]
- hardware: GPU 20 [p.2], NVIDIA A100 3 [p.9], Intel Xeon 2 [p.9], AMD EPYC 1 [p.9]
- technique: Convolution 26 [p.1], Bootstrapping 20 [p.1], Quantization 19 [p.4], Key switching 9 [p.3], ReLU 9 [p.2], Compiler 7 [p.12], Softmax 4 [p.2], Semi-honest model 3 [p.4], Polynomial approximation 2 [p.4], SIMD packing 2 [p.1], Two-party computation 2 [p.13]
- assumption: RLWE 30 [p.2], LWE 12 [p.3], Lattice-based 2 [p.15]
- task: Medical / healthcare 3 [p.1], Finance 3 [p.1], Genomics 2 [p.13]
- topic: Homomorphic encryption 56 [p.1], Private / secure inference 3 [p.1]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/ppml/inference/biswas-2025_practical-and-private-hybrid-ml-inference-with-fhe.pdf |
ppml/inference | 17 | 4.2 MB | 5d58be441ee5 |
was Biswas et al. - 2025 - Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption.pdf |
Database key: practical-and-private-hybrid-ml-inference-with-fully-homomorphic · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set practical-and-private-hybrid-ml-inference-with-fully-homomorphic COLUMN VALUE