Skip to content

Secure human action recognition by encrypted neural network inference

Authors: Miran Kim; Xiaoqian Jiang; Kristin Lauter; Elkhan Ismayilzada; Shayan Shams
Year: 2022
Venue: Nature Communications

type: application collection: ppml status: auto

Links: DOI

Summary

Auto-extracted summary (unreviewed)

HEAR is a cloud-based secure human action recognition system in which skeleton joints extracted from home-monitoring video are encrypted under CKKS and classified by a CNN evaluated homomorphically on an untrusted cloud. Its main technical contribution is a fast homomorphic convolution that exploits the sparsity of ciphertexts after average pooling by interlacing several sparsely packed channels into one ciphertext. The protocol distinguishes falls from daily activities with 86.21% sensitivity and 99.14% specificity at 1.2 s (small net) and 2.4 s (large net) average latency, giving a 613x speedup over LoLa and 3.1x higher throughput than nGraph-HE2.

Abstract

Advanced computer vision technology can provide near real-time home monitoring to support “aging in place” by detecting falls and symptoms related to seizures and stroke. Affordable webcams, together with cloud computing services (to run machine learning algorithms), can potentially bring significant social benefits. However, it has not been deployed in practice because of privacy concerns. In this paper, we propose a strategy that uses homomorphic encryption to resolve this dilemma, which guarantees information confidentiality while retaining action detection. Our protocol for secure inference can distinguish falls from activities of daily living with 86.21% sensitivity and 99.14% specificity, with an average inference latency of 1.2 seconds and 2.4 seconds on real-world test datasets using small and large neural nets, respectively. We show that our method enables a 613x speedup over the latency-optimized LoLa and achieves an average of 3.1x throughput increase in secure inference compared to the throughput-optimized nGraph-HE2.
Source: crossref

Extracted details

Field Value Source Confidence
contribution HEAR: FHE-based secure service paradigm for cloud action recognition from encrypted skeleton data llm 0.85
contribution Fast homomorphic multi-channel convolution exploiting sparsely packed ciphertexts via interlaced packing llm 0.85
contribution Full-pipeline evaluation on real-world fall vs. activities-of-daily-living data with small and large CNNs llm 0.85
contribution Open-source implementation based on a modified Microsoft SEAL llm 0.85
schemes CKKS llm 0.85
libraries SEAL llm 0.85
libraries PyTorch llm 0.85
models CNN (small and large variants) llm 0.85
models HRNet (plaintext pose estimation) llm 0.85
datasets real-world fall / activities-of-daily-living test datasets llm 0.85
datasets MPII Human Pose (pretrained pose estimator) llm 0.85
tasks human action recognition llm 0.85
tasks fall detection llm 0.85
tasks privacy-preserving home monitoring / telemedicine llm 0.85
security level 128-bit llm 0.85
threat model honest-but-curious cloud server; three parties (monitoring service provider holds secret key, end-users encrypt, cloud evaluates) llm 0.85
interaction non-interactive FHE llm 0.85
code url https://github.com/K-miran/HEAR51 (URL found in the full text) regex 0.50
code url https://github.com/K-miran/HEAR51.Article (URL found in the full text) regex 0.50
code url https://github.com/Microsoft/SEAL (URL found in the full text) regex 0.50
code url https://github.com/K-miran/HEAR llm 0.85
llm relevance medium llm 0.85
key result sensitivity: 86.21% (fall vs. activities of daily living, encrypted inference) llm 0.85
key result specificity: 99.14% (fall vs. activities of daily living, encrypted inference) llm 0.85
key result average inference latency: 1.2 s / 2.4 s (small / large neural net) llm 0.85
key result speedup over LoLa: 613x (latency-optimized baseline) llm 0.85
key result throughput increase over nGraph-HE2: 3.1x average (throughput-optimized baseline) llm 0.85

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: MPC 7 [p.6], CKKS 4 [p.8], Leveled / somewhat HE 3 [p.11], TFHE 2 [p.11]
  • library: nGraph-HE 19 [p.1], Microsoft SEAL 3 [p.11], EVA / CHET 3 [p.11], HElib 1 [p.13]
  • model: CNN 50 [p.2], ResNet 1 [p.4], RNN / LSTM 1 [p.12], Logistic regression 1 [p.12]
  • dataset: MNIST 2 [p.5], Action recognition (UCF/HMDB/KTH) 2 [p.3], CIFAR-10 1 [p.11]
  • hardware: NVIDIA V100 1 [p.3], Intel Xeon 1 [p.11], GPU 1 [p.3]
  • technique: Convolution 99 [p.2], Rotations 28 [p.2], SIMD packing 13 [p.2], RNS 4 [p.10], Rescaling 3 [p.11], NTT 3 [p.11], Compiler 3 [p.12], Bootstrapping 2 [p.12], ReLU 2 [p.4], Semi-honest model 2 [p.2]
  • assumption: LWE 4 [p.11], IND-CPA 2 [p.8]
  • task: Medical / healthcare 13 [p.1], Genomics 8 [p.8], IoT / edge 2 [p.1]
  • topic: Homomorphic encryption 46 [p.1], Private / secure inference 23 [p.1], Benchmark 4 [p.2], Open source 3 [p.3]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/ppml/inference/kim-2022_secure-human-action-recognition-by-encrypted-neural-network-inference.pdf ppml/inference 13 2.9 MB 3d446c6c19a8 was Kim et al. - 2022 - Secure human action recognition by encrypted neural network inference.pdf

Database key: secure-human-action-recognition-by-encrypted-neural-network-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set secure-human-action-recognition-by-encrypted-neural-network-inference COLUMN VALUE