Skip to content

Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption

Authors: Sayan Biswas; Philippe Chartier; Akash Dhasade; Tom Jurien; David Kerriou; Anne-Marie Kerrmarec; Mohammed Lemou; Franklin Tranie; Martijn de Vos; Milos Vujasinovic
Year: 2025
Venue: arXiv preprint

type: system collection: ppml status: auto

Links: arXiv:2509.01253

Summary

Auto-extracted summary (unreviewed)

Safhire is a hybrid FHE inference framework in which the server evaluates linear layers under an RLWE-based encryption while the client computes exact non-linear activations in plaintext, which removes bootstrapping and activation approximations; randomized shuffling of intermediate values protects model confidentiality, and fast ciphertext packing plus partial trace extraction reduce latency. Implemented in Julia with CPU and GPU support, it achieves up to 7.6x lower latency than Orion and 7.4x lower than Concrete-ML on ResNet-20/18/34 over CIFAR-10, Tiny and ImageNet with comparable accuracy and manageable communication.

Abstract

In contemporary cloud-based services, protecting users' sensitive data and ensuring the confidentiality of the server's model are critical. Fully homomorphic encryption (FHE) enables inference directly on encrypted inputs, but its practicality is hindered by expensive bootstrapping and inefficient approximations of non-linear activations. We introduce Safhire, a hybrid inference framework that executes linear layers under encryption on the server while offloading non- linearities to the client in plaintext. This design eliminates bootstrapping, supports exact activations, and significantly reduces computation cost. To safeguard model confidentiality despite client access to intermediate outputs, Safhire applies randomized shuffling, which obfuscates intermediate values and makes it practically impossible for a client to reconstruct the model. To further reduce inference latency, Safhire incorporates advanced optimizations such as fast ciphertext packing and partial extraction. Evaluations on multiple standard models and datasets show that Safhire achieves up to 7.6x and 7.4x lower inference latency than Orion and Concrete-ML, respectively, which are two state-of-the-art FHE baselines, with manageable communication overhead and comparable accuracy. These results underline the practicality of our hybrid FHE inference scheme.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution Hybrid client-server FHE inference that eliminates bootstrapping by offloading non-linearities to the client llm 0.85
contribution Randomized shuffling of intermediate outputs to protect model confidentiality llm 0.85
contribution Fast ciphertext packing and partial trace extraction optimizations llm 0.85
contribution Multi-threaded CPU and GPU implementation in Julia llm 0.85
schemes RLWE-based leveled HE (custom) llm 0.85
libraries Julia llm 0.85
libraries NNlib.jl llm 0.85
libraries FFTW.jl llm 0.85
libraries Permutations.jl llm 0.85
libraries Orion (baseline) llm 0.85
libraries Concrete ML (baseline) llm 0.85
models ResNet-20 llm 0.85
models ResNet-18 llm 0.85
models ResNet-34 llm 0.85
datasets CIFAR-10 llm 0.85
datasets Tiny ImageNet llm 0.85
datasets ImageNet llm 0.85
tasks image classification llm 0.85
tasks private CNN inference llm 0.85
hardware 2x Intel Xeon E5-2690 v4 (56 logical cores, 925 GB RAM) llm 0.85
hardware 2x Intel Xeon Platinum 8272L (104 logical cores, 2975 GB RAM) llm 0.85
hardware NVIDIA A100-SXM4 80GB GPU with 24-core AMD EPYC 7543 llm 0.85
security level at least 128-bit (Albrecht LWE estimator) llm 0.85
threat model client-server; client sees shuffled intermediate activations; model confidentiality via shuffling llm 0.85
interaction interactive (HE+MPC) llm 0.85
limitations Client must participate in every non-linear layer, adding communication overhead llm 0.85
limitations Max pooling replaced by average pooling for ResNet-34 on ImageNet llm 0.85
llm relevance medium llm 0.85
key result inference latency speed-up vs Orion: up to 7.6x (1.5x to 10.5x range) (single-threaded CPU, ResNet models) llm 0.85
key result inference latency speed-up vs Concrete-ML: up to 7.4x (FHE baselines comparison) llm 0.85
key result Orion baseline latency (motivation): 17 min / 80 min / 180 min (ResNet-20 CIFAR-10 / ResNet-18 Tiny / ResNet-34 ImageNet, bootstrapping 62.7 to 82.2% of time) llm 0.85
key result correctness: no discrepancies vs quantized cleartext model (all tested precision and trace-extraction configurations) llm 0.85

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 5 [p.9], TFHE 5 [p.2], Differential privacy 5 [p.2], Multiparty HE 3 [p.13], MPC 3 [p.13], Leveled / somewhat HE 2 [p.12]
  • library: Zama 2 [p.13], Orion 2 [p.9], EVA / CHET 2 [p.12], Concrete (Zama) 1 [p.14]
  • model: ResNet 45 [p.1], CNN 11 [p.1], GPT 1 [p.13]
  • dataset: CIFAR-10 18 [p.1], ImageNet 13 [p.1], MNIST 1 [p.12]
  • hardware: GPU 20 [p.2], NVIDIA A100 3 [p.9], Intel Xeon 2 [p.9], AMD EPYC 1 [p.9]
  • technique: Convolution 26 [p.1], Bootstrapping 20 [p.1], Quantization 19 [p.4], Key switching 9 [p.3], ReLU 9 [p.2], Compiler 7 [p.12], Softmax 4 [p.2], Semi-honest model 3 [p.4], Polynomial approximation 2 [p.4], SIMD packing 2 [p.1], Two-party computation 2 [p.13]
  • assumption: RLWE 30 [p.2], LWE 12 [p.3], Lattice-based 2 [p.15]
  • task: Medical / healthcare 3 [p.1], Finance 3 [p.1], Genomics 2 [p.13]
  • topic: Homomorphic encryption 56 [p.1], Private / secure inference 3 [p.1]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/ppml/inference/biswas-2025_practical-and-private-hybrid-ml-inference-with-fhe.pdf ppml/inference 17 4.2 MB 5d58be441ee5 was Biswas et al. - 2025 - Practical and Private Hybrid ML Inference with Fully Homomorphic Encryption.pdf

Database key: practical-and-private-hybrid-ml-inference-with-fully-homomorphic · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set practical-and-private-hybrid-ml-inference-with-fully-homomorphic COLUMN VALUE