Activate Me!: Designing Efficient Activation Functions for Privacy-Preserving Machine Learning with Fully Homomorphic Encryption¶
Authors: Nges Brian Njungle; Michel A. Kinsy
Year: 2026
Venue: Lecture Notes in Computer Science
type: technique collection: ppml status: auto
Links: DOI
Summary¶
Auto-extracted summary (unreviewed)
Njungle and Kinsy design and benchmark activation functions for CKKS-based encrypted neural network inference in OpenFHE, comparing the square function, a low-degree polynomial approximation of ReLU, and a new ReLU evaluated by scheme switching between CKKS and TFHE. On LeNet-5 (MNIST) the square activation reaches 99.4% accuracy at 128 s per image without bootstrapping, while on ResNet-20 (CIFAR-10) polynomial ReLU gives 83.8% at 1145 s per image and scheme-switching ReLU gives 89.8% at 1697 s per image, exposing a direct accuracy-versus-latency trade-off.
Abstract¶
The rapid integration of machine learning into applications in sensitive domains like healthcare and defense raises serious privacy and security concerns. These applications require strong privacy protections, as they rely on large amounts of sensitive data for both training and inference. Fully Homomorphic Encryption (FHE) offers a promising solution by allowing computations on encrypted data, thereby preserving confidentiality throughout the entire machine-learning pipeline. However, FHE only supports linear operations natively. This makes it hard to implement non-linear activation functions - an essential component of modern machine learning applications - under FHE constraints. In this study, we design, implement, and evaluate activation functions optimized for FHE-based machine learning applications. We focus on two widely used functions: the Square function and the Rectified Linear Unit (ReLU). Our experiments utilize the LeNet-5 and ResNet-20 architectures implemented with the CKKS scheme from the OpenFHE library. For ReLU, we compare two approaches. First, we explore the popular low-degree polynomial approximation approach. Second, we introduce a novel scheme-switching technique that also securely evaluates ReLU under FHE constraints. Our results show that the square function is highly effective in shallow models like LeNet-5, achieving 99.4% accuracy with an inference time of 128 s per image. In deeper networks like ResNet- 20, ReLU is more appropriate. The FHE-based ResNet-20 model implemented with ReLU polynomial approximation resulted in an accuracy of 83.8% and an inference time of 1,145 s per image. Further, our proposed scheme-switching algorithm achieved a higher accuracy of 89.8%, with an increased inference time of 1,697 s per image on a ResNet-20 model as well. These results highlight the key trade-off to be considered when selecting activation functions for FHE-based machine learning applications. The activation functions that reduce computation time tend to significantly lower accuracy, while those that preserve accuracy incur higher computational costs.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | design and evaluation of FHE-friendly activation functions (square, polynomial ReLU) | llm | 0.90 |
| contribution | novel scheme-switching (CKKS to TFHE) algorithm for high-precision ReLU under FHE | llm | 0.90 |
| contribution | bootstrapping-free LeNet-5 inference with square activation | llm | 0.90 |
| contribution | accuracy/latency trade-off analysis on LeNet-5 and ResNet-20 | llm | 0.90 |
| schemes | CKKS | llm | 0.90 |
| schemes | TFHE | llm | 0.90 |
| libraries | OpenFHE | llm | 0.90 |
| libraries | PyTorch | llm | 0.90 |
| models | LeNet-5 | llm | 0.90 |
| models | ResNet-20 | llm | 0.90 |
| datasets | MNIST | llm | 0.90 |
| datasets | CIFAR-10 | llm | 0.90 |
| tasks | private image classification inference | llm | 0.90 |
| hardware | AMD Ryzen 9 5900X 12-core CPU, 64 GB RAM | llm | 0.90 |
| security level | around 128-bit | llm | 0.90 |
| interaction | non-interactive FHE | llm | 0.90 |
| limitations | square activation is infeasible for training deep networks such as ResNet-20 | llm | 0.90 |
| limitations | polynomial ReLU approximation causes an 8.4% accuracy drop on ResNet-20 | llm | 0.90 |
| limitations | scheme-switching ReLU preserves accuracy (2.4% drop) but increases latency | llm | 0.90 |
| llm relevance | medium | llm | 0.90 |
| key result | accuracy: 99.4% (LeNet-5, MNIST, square activation, CKKS) | llm | 0.90 |
| key result | inference latency: 128 s per image (LeNet-5, MNIST, square activation, CKKS) | llm | 0.90 |
| key result | accuracy: 83.8% (ResNet-20, CIFAR-10, polynomial ReLU approximation) | llm | 0.90 |
| key result | inference latency: 1,145 s per image (ResNet-20, CIFAR-10, polynomial ReLU approximation) | llm | 0.90 |
| key result | accuracy: 89.8% (ResNet-20, CIFAR-10, scheme-switching ReLU) | llm | 0.90 |
| key result | inference latency: 1,697 s per image (ResNet-20, CIFAR-10, scheme-switching ReLU) | llm | 0.90 |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 31 [p.1], Scheme switching 26 [p.1], TFHE 23 [p.5], FHEW 3 [p.12], MPC 3 [p.4], BGV 2 [p.5]
- library: OpenFHE 5 [p.1], PyTorch 1 [p.14]
- model: ResNet 32 [p.1], LeNet 31 [p.1], CNN 4 [p.5], AlexNet 1 [p.4]
- dataset: CIFAR-10 6 [p.5], MNIST 3 [p.4]
- hardware: GPU 2 [p.4]
- technique: ReLU 80 [p.1], Bootstrapping 19 [p.4], Square activation 16 [p.4], Polynomial approximation 15 [p.1], Convolution 13 [p.8], Chebyshev 11 [p.10], Sigmoid 5 [p.8], SIMD packing 4 [p.7]
- assumption: LWE 6 [p.7], RLWE 4 [p.7], Lattice-based 4 [p.6]
- task: Medical / healthcare 4 [p.1]
- topic: Homomorphic encryption 130 [p.1], Privacy-preserving machine learning 30 [p.1], Private / secure inference 7 [p.3], Benchmark 2 [p.15], Open source 2 [p.20]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/ppml/inference/njungle-2025_activate-me-efficient-activation-functions-for-ppml-with-fhe.pdf |
ppml/inference | 23 | 613.9 KB | 405b0a836728 |
was Njungle and Kinsy - 2025 - Activate Me! Designing Efficient Activation Functions for Privacy-Preserving Machine Learning with.pdf |
Database key: activate-me-designing-efficient-activation-functions-for-privacy · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set activate-me-designing-efficient-activation-functions-for-privacy COLUMN VALUE