Skip to content

Activate Me!: Designing Efficient Activation Functions for Privacy-Preserving Machine Learning with Fully Homomorphic Encryption

Authors: Nges Brian Njungle; Michel A. Kinsy
Year: 2026
Venue: Lecture Notes in Computer Science

type: technique collection: ppml status: auto

Links: DOI

Summary

Auto-extracted summary (unreviewed)

Njungle and Kinsy design and benchmark activation functions for CKKS-based encrypted neural network inference in OpenFHE, comparing the square function, a low-degree polynomial approximation of ReLU, and a new ReLU evaluated by scheme switching between CKKS and TFHE. On LeNet-5 (MNIST) the square activation reaches 99.4% accuracy at 128 s per image without bootstrapping, while on ResNet-20 (CIFAR-10) polynomial ReLU gives 83.8% at 1145 s per image and scheme-switching ReLU gives 89.8% at 1697 s per image, exposing a direct accuracy-versus-latency trade-off.

Abstract

The rapid integration of machine learning into applications in sensitive domains like healthcare and defense raises serious privacy and security concerns. These applications require strong privacy protections, as they rely on large amounts of sensitive data for both training and inference. Fully Homomorphic Encryption (FHE) offers a promising solution by allowing computations on encrypted data, thereby preserving confidentiality throughout the entire machine-learning pipeline. However, FHE only supports linear operations natively. This makes it hard to implement non-linear activation functions - an essential component of modern machine learning applications - under FHE constraints. In this study, we design, implement, and evaluate activation functions optimized for FHE-based machine learning applications. We focus on two widely used functions: the Square function and the Rectified Linear Unit (ReLU). Our experiments utilize the LeNet-5 and ResNet-20 architectures implemented with the CKKS scheme from the OpenFHE library. For ReLU, we compare two approaches. First, we explore the popular low-degree polynomial approximation approach. Second, we introduce a novel scheme-switching technique that also securely evaluates ReLU under FHE constraints. Our results show that the square function is highly effective in shallow models like LeNet-5, achieving 99.4% accuracy with an inference time of 128 s per image. In deeper networks like ResNet- 20, ReLU is more appropriate. The FHE-based ResNet-20 model implemented with ReLU polynomial approximation resulted in an accuracy of 83.8% and an inference time of 1,145 s per image. Further, our proposed scheme-switching algorithm achieved a higher accuracy of 89.8%, with an increased inference time of 1,697 s per image on a ResNet-20 model as well. These results highlight the key trade-off to be considered when selecting activation functions for FHE-based machine learning applications. The activation functions that reduce computation time tend to significantly lower accuracy, while those that preserve accuracy incur higher computational costs.
Source: pdf

Extracted details

Field Value Source Confidence
contribution design and evaluation of FHE-friendly activation functions (square, polynomial ReLU) llm 0.90
contribution novel scheme-switching (CKKS to TFHE) algorithm for high-precision ReLU under FHE llm 0.90
contribution bootstrapping-free LeNet-5 inference with square activation llm 0.90
contribution accuracy/latency trade-off analysis on LeNet-5 and ResNet-20 llm 0.90
schemes CKKS llm 0.90
schemes TFHE llm 0.90
libraries OpenFHE llm 0.90
libraries PyTorch llm 0.90
models LeNet-5 llm 0.90
models ResNet-20 llm 0.90
datasets MNIST llm 0.90
datasets CIFAR-10 llm 0.90
tasks private image classification inference llm 0.90
hardware AMD Ryzen 9 5900X 12-core CPU, 64 GB RAM llm 0.90
security level around 128-bit llm 0.90
interaction non-interactive FHE llm 0.90
limitations square activation is infeasible for training deep networks such as ResNet-20 llm 0.90
limitations polynomial ReLU approximation causes an 8.4% accuracy drop on ResNet-20 llm 0.90
limitations scheme-switching ReLU preserves accuracy (2.4% drop) but increases latency llm 0.90
llm relevance medium llm 0.90
key result accuracy: 99.4% (LeNet-5, MNIST, square activation, CKKS) llm 0.90
key result inference latency: 128 s per image (LeNet-5, MNIST, square activation, CKKS) llm 0.90
key result accuracy: 83.8% (ResNet-20, CIFAR-10, polynomial ReLU approximation) llm 0.90
key result inference latency: 1,145 s per image (ResNet-20, CIFAR-10, polynomial ReLU approximation) llm 0.90
key result accuracy: 89.8% (ResNet-20, CIFAR-10, scheme-switching ReLU) llm 0.90
key result inference latency: 1,697 s per image (ResNet-20, CIFAR-10, scheme-switching ReLU) llm 0.90

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: CKKS 31 [p.1], Scheme switching 26 [p.1], TFHE 23 [p.5], FHEW 3 [p.12], MPC 3 [p.4], BGV 2 [p.5]
  • library: OpenFHE 5 [p.1], PyTorch 1 [p.14]
  • model: ResNet 32 [p.1], LeNet 31 [p.1], CNN 4 [p.5], AlexNet 1 [p.4]
  • dataset: CIFAR-10 6 [p.5], MNIST 3 [p.4]
  • hardware: GPU 2 [p.4]
  • technique: ReLU 80 [p.1], Bootstrapping 19 [p.4], Square activation 16 [p.4], Polynomial approximation 15 [p.1], Convolution 13 [p.8], Chebyshev 11 [p.10], Sigmoid 5 [p.8], SIMD packing 4 [p.7]
  • assumption: LWE 6 [p.7], RLWE 4 [p.7], Lattice-based 4 [p.6]
  • task: Medical / healthcare 4 [p.1]
  • topic: Homomorphic encryption 130 [p.1], Privacy-preserving machine learning 30 [p.1], Private / secure inference 7 [p.3], Benchmark 2 [p.15], Open source 2 [p.20]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/ppml/inference/njungle-2025_activate-me-efficient-activation-functions-for-ppml-with-fhe.pdf ppml/inference 23 613.9 KB 405b0a836728 was Njungle and Kinsy - 2025 - Activate Me! Designing Efficient Activation Functions for Privacy-Preserving Machine Learning with.pdf

Database key: activate-me-designing-efficient-activation-functions-for-privacy · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set activate-me-designing-efficient-activation-functions-for-privacy COLUMN VALUE