Privacy-Preserving Machine Learning With Fully Homomorphic Encryption for Deep Neural Network¶
Authors: Joon-Woo Lee; Hyungchul Kang; Yongwoo Lee; Woosuk Choi; Jieun Eom; Maxim Deryabin; Eunsang Lee; Junghyun Lee; Donghoon Yoo; Young-Sik Kim; Jong-Seon No
Year: 2022
Venue: IEEE Access
type: system collection: ppml status: auto
Links: DOI
Summary¶
Auto-extracted summary (unreviewed)
Implements the standard ResNet-20 model on encrypted CIFAR-10 images with the RNS-CKKS scheme, using high-precision polynomial approximations of ReLU and softmax instead of replacing them, and for the first time applying CKKS bootstrapping so that arbitrarily deep networks can be evaluated. The encrypted model agrees with the plaintext ResNet-20 on 98.43% of inputs and reaches 92.43% (+/-2.65%) accuracy versus 91.89% in the clear, with one inference taking about 3 hours on a dual Xeon Platinum 8280 (112 cores) at 111.6-bit security.
Abstract¶
Fully homomorphic encryption (FHE) is a prospective tool for privacy-preserving machine learning (PPML). Several PPML models have been proposed based on various FHE schemes and approaches. Although FHE schemes are suitable as tools for implementing PPML models, previous PPML models based on FHE, such as CryptoNet, SEALion, and CryptoDL, are limited to simple and nonstandard types of machine learning models; they have not proven to be efficient and accurate with more practical and advanced datasets. Previous PPML schemes replaced non-arithmetic activation functions with simple arithmetic functions instead of adopting approximation methods and did not use bootstrapping, which enables continuous homomorphic evaluations. Thus, they could neither use standard activation functions nor employ large numbers of layers. In this work, we first implement the standard ResNet-20 model with the RNS-CKKS FHE with bootstrapping and verify the implemented model with the CIFAR-10 dataset and plaintext model parameters. Instead of replacing the non-arithmetic functions with simple arithmetic functions, we use state-of-the-art approximation methods to evaluate these non-arithmetic functions, such as ReLU and Softmax, with sufficient precision. Further, for the first time, we use the bootstrapping technique of the RNS-CKKS scheme in the proposed model, which enables us to evaluate an arbitrary deep learning model on encrypted data. We numerically verify that the proposed model with the CIFAR-10 dataset shows 98.43% identical results to the original ResNet-20 model with non-encrypted data. The classification accuracy of the proposed model is 92.43%±2.65%, which is quite close to that of the original ResNet-20 CNN model (91.89%). It takes approximately 3 h for inference on a dual Intel Xeon Platinum 8280 CPU (112 cores) with 172 GB of memory. We believe that this opens the possibility of applying FHE to an advanced deep PPML model.
Source: pdf
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | First standard ResNet-20 evaluation under RNS-CKKS with bootstrapping | llm | 0.85 |
| contribution | Use of precise polynomial approximations for ReLU and softmax rather than function replacement | llm | 0.85 |
| contribution | Demonstration that bootstrapping enables arbitrarily deep encrypted inference | llm | 0.85 |
| schemes | CKKS | llm | 0.85 |
| libraries | SEAL | llm | 0.85 |
| models | ResNet-20 | llm | 0.85 |
| datasets | CIFAR-10 | llm | 0.85 |
| tasks | image classification | llm | 0.85 |
| tasks | private CNN inference | llm | 0.85 |
| hardware | dual Intel Xeon Platinum 8280 CPU (112 cores), 172 GB memory | llm | 0.85 |
| security level | 111.6-bit | llm | 0.85 |
| threat model | client-server; IND-CPA security of FHE | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| code url | https://github.com/microsoft/SEAL (URL found in the full text) | regex | 0.50 |
| limitations | Security level of 111.6 bits is below the standard 128 bits and may be regarded as insecure | llm | 0.85 |
| limitations | Inference takes about 3 hours per image | llm | 0.85 |
| llm relevance | medium | llm | 0.85 |
| key result | inference latency: about 3 h (ResNet-20, CIFAR-10, dual Xeon Platinum 8280, 112 cores) | llm | 0.85 |
| key result | classification accuracy: 92.43% +/- 2.65% (plaintext model 91.89%) (CIFAR-10) | llm | 0.85 |
| key result | agreement with plaintext model: 98.43% identical results (CIFAR-10) | llm | 0.85 |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: CKKS 55 [p.1], Leveled / somewhat HE 6 [p.3], TFHE 4 [p.2], MPC 4 [p.2], RSA 3 [p.14], BFV 2 [p.2]
- library: Microsoft SEAL 10 [p.1], nGraph-HE 1 [p.14]
- model: ResNet 62 [p.1], CNN 6 [p.1], AlexNet 1 [p.14], VGG 1 [p.3]
- dataset: CIFAR-10 12 [p.1], ImageNet 3 [p.2], MNIST 1 [p.2]
- hardware: GPU 3 [p.13], Intel Xeon 2 [p.1], NVIDIA T4 1 [p.6], FPGA 1 [p.13], ASIC 1 [p.13]
- technique: Bootstrapping 81 [p.1], Convolution 61 [p.2], RNS 41 [p.1], Softmax 36 [p.1], ReLU 31 [p.1], Rotations 16 [p.3], Minimax approximation 15 [p.4], Key switching 12 [p.7], Goldschmidt 8 [p.7], Rescaling 5 [p.3], Polynomial approximation 5 [p.7], Chebyshev 5 [p.5], Remez 4 [p.4], Model extraction / stealing 3 [p.2], Relinearization 2 [p.9], NTT 2 [p.9]
- assumption: Lattice-based 5 [p.14], LWE 4 [p.3], RLWE 2 [p.3]
- task: Genomics 2 [p.15]
- topic: Homomorphic encryption 69 [p.1], Privacy-preserving machine learning 45 [p.1], Private / secure inference 2 [p.14]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/ppml/inference/lee-2021_privacy-preserving-ml-with-fhe-for-deep-neural-network.pdf |
ppml/inference | 16 | 1.4 MB | fd54edeebc2f |
was Lee et al. - 2021 - Privacy-Preserving Machine Learning With Fully Homomorphic Encryption for Deep Neural Network.pdf |
Database key: privacy-preserving-machine-learning-with-fully-homomorphic-encryption · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set privacy-preserving-machine-learning-with-fully-homomorphic-encryption COLUMN VALUE