Secure human action recognition by encrypted neural network inference¶
Authors: Miran Kim; Xiaoqian Jiang; Kristin Lauter; Elkhan Ismayilzada; Shayan Shams
Year: 2022
Venue: Nature Communications
type: application collection: ppml status: auto
Links: DOI
Summary¶
Auto-extracted summary (unreviewed)
HEAR is a cloud-based secure human action recognition system in which skeleton joints extracted from home-monitoring video are encrypted under CKKS and classified by a CNN evaluated homomorphically on an untrusted cloud. Its main technical contribution is a fast homomorphic convolution that exploits the sparsity of ciphertexts after average pooling by interlacing several sparsely packed channels into one ciphertext. The protocol distinguishes falls from daily activities with 86.21% sensitivity and 99.14% specificity at 1.2 s (small net) and 2.4 s (large net) average latency, giving a 613x speedup over LoLa and 3.1x higher throughput than nGraph-HE2.
Abstract¶
Advanced computer vision technology can provide near real-time home monitoring to support “aging in place” by detecting falls and symptoms related to seizures and stroke. Affordable webcams, together with cloud computing services (to run machine learning algorithms), can potentially bring significant social benefits. However, it has not been deployed in practice because of privacy concerns. In this paper, we propose a strategy that uses homomorphic encryption to resolve this dilemma, which guarantees information confidentiality while retaining action detection. Our protocol for secure inference can distinguish falls from activities of daily living with 86.21% sensitivity and 99.14% specificity, with an average inference latency of 1.2 seconds and 2.4 seconds on real-world test datasets using small and large neural nets, respectively. We show that our method enables a 613x speedup over the latency-optimized LoLa and achieves an average of 3.1x throughput increase in secure inference compared to the throughput-optimized nGraph-HE2.
Source: crossref
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | HEAR: FHE-based secure service paradigm for cloud action recognition from encrypted skeleton data | llm | 0.85 |
| contribution | Fast homomorphic multi-channel convolution exploiting sparsely packed ciphertexts via interlaced packing | llm | 0.85 |
| contribution | Full-pipeline evaluation on real-world fall vs. activities-of-daily-living data with small and large CNNs | llm | 0.85 |
| contribution | Open-source implementation based on a modified Microsoft SEAL | llm | 0.85 |
| schemes | CKKS | llm | 0.85 |
| libraries | SEAL | llm | 0.85 |
| libraries | PyTorch | llm | 0.85 |
| models | CNN (small and large variants) | llm | 0.85 |
| models | HRNet (plaintext pose estimation) | llm | 0.85 |
| datasets | real-world fall / activities-of-daily-living test datasets | llm | 0.85 |
| datasets | MPII Human Pose (pretrained pose estimator) | llm | 0.85 |
| tasks | human action recognition | llm | 0.85 |
| tasks | fall detection | llm | 0.85 |
| tasks | privacy-preserving home monitoring / telemedicine | llm | 0.85 |
| security level | 128-bit | llm | 0.85 |
| threat model | honest-but-curious cloud server; three parties (monitoring service provider holds secret key, end-users encrypt, cloud evaluates) | llm | 0.85 |
| interaction | non-interactive FHE | llm | 0.85 |
| code url | https://github.com/K-miran/HEAR51 (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/K-miran/HEAR51.Article (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/Microsoft/SEAL (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/K-miran/HEAR | llm | 0.85 |
| llm relevance | medium | llm | 0.85 |
| key result | sensitivity: 86.21% (fall vs. activities of daily living, encrypted inference) | llm | 0.85 |
| key result | specificity: 99.14% (fall vs. activities of daily living, encrypted inference) | llm | 0.85 |
| key result | average inference latency: 1.2 s / 2.4 s (small / large neural net) | llm | 0.85 |
| key result | speedup over LoLa: 613x (latency-optimized baseline) | llm | 0.85 |
| key result | throughput increase over nGraph-HE2: 3.1x average (throughput-optimized baseline) | llm | 0.85 |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- scheme: MPC 7 [p.6], CKKS 4 [p.8], Leveled / somewhat HE 3 [p.11], TFHE 2 [p.11]
- library: nGraph-HE 19 [p.1], Microsoft SEAL 3 [p.11], EVA / CHET 3 [p.11], HElib 1 [p.13]
- model: CNN 50 [p.2], ResNet 1 [p.4], RNN / LSTM 1 [p.12], Logistic regression 1 [p.12]
- dataset: MNIST 2 [p.5], Action recognition (UCF/HMDB/KTH) 2 [p.3], CIFAR-10 1 [p.11]
- hardware: NVIDIA V100 1 [p.3], Intel Xeon 1 [p.11], GPU 1 [p.3]
- technique: Convolution 99 [p.2], Rotations 28 [p.2], SIMD packing 13 [p.2], RNS 4 [p.10], Rescaling 3 [p.11], NTT 3 [p.11], Compiler 3 [p.12], Bootstrapping 2 [p.12], ReLU 2 [p.4], Semi-honest model 2 [p.2]
- assumption: LWE 4 [p.11], IND-CPA 2 [p.8]
- task: Medical / healthcare 13 [p.1], Genomics 8 [p.8], IoT / edge 2 [p.1]
- topic: Homomorphic encryption 46 [p.1], Private / secure inference 23 [p.1], Benchmark 4 [p.2], Open source 3 [p.3]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/ppml/inference/kim-2022_secure-human-action-recognition-by-encrypted-neural-network-inference.pdf |
ppml/inference | 13 | 2.9 MB | 3d446c6c19a8 |
was Kim et al. - 2022 - Secure human action recognition by encrypted neural network inference.pdf |
Database key: secure-human-action-recognition-by-encrypted-neural-network-inference · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set secure-human-action-recognition-by-encrypted-neural-network-inference COLUMN VALUE