Skip to content

PrivateLoRA for Efficient Privacy Preserving LLM

Authors: Yiming Wang; Yu Lin; Xiaodong Zeng; Guannan Zhang
Year: 2023
Venue: arXiv 2023

type: system collection: catalogue status: auto

Links: arXiv:2311.14030

Summary

Auto-extracted summary (unreviewed)

PrivateLoRA proposes a heterogeneous LLM serving paradigm in which privacy-sensitive LoRA-style computation stays on the user's edge device while the shared decoder stack runs in the cloud, exchanging only activations. By exploiting the low rank of residual activations it cuts communication by over 95%, reaching more than 300% of device-only throughput for 7B models and over 80% of A100 throughput for 33B models under 5G, with tuning quality comparable to LoRA.

Abstract

End users face a choice between privacy and efficiency in current Large Language Model (LLM) service paradigms. In cloud-based paradigms, users are forced to compromise data locality for generation quality and processing speed. Conversely, edge device paradigms maintain data locality but fail to deliver satisfactory performance. In this work, we propose a novel LLM service paradigm that distributes privacy-sensitive computation on edge devices and shared computation in the cloud. Only activations are transmitted between the central cloud and edge devices to ensure data locality. Our core innovation, PrivateLoRA, addresses the challenging communication overhead by exploiting the low rank of residual activations, achieving over 95% communication reduction. Consequently, PrivateLoRA effectively maintains data locality and is extremely resource efficient. Under standard 5G networks, PrivateLoRA achieves throughput over 300% of device-only solutions for 7B models and over 80% of an A100 GPU for 33B models. PrivateLoRA also provides tuning performance comparable to LoRA for advanced personalization. Our approach democratizes access to state-of-the-art generative AI for edge devices, paving the way for more tailored LLM experiences for the general public. To our knowledge, our proposed framework is the first efficient and privacy-preserving LLM solution in the literature.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution device-cloud distributed LLM serving paradigm preserving data locality (only activations transmitted) llm 0.80
contribution PrivateLoRA: low-rank residual transmission reducing communication by over 95% llm 0.80
contribution throughput analysis showing >300% of device-only speed for 7B and >80% of A100 for 33B models under 5G llm 0.80
contribution tuning performance comparable to LoRA for personalization llm 0.80
libraries LM-Eval Harness llm 0.80
libraries GPTQ llm 0.80
libraries AutoGPTQ llm 0.80
libraries llama.cpp llm 0.80
libraries PEFT llm 0.80
models LLaMA-2-7B llm 0.80
models LLaMA-2-13B llm 0.80
models 33B LLaMA model llm 0.80
models OPT-1.3B llm 0.80
models Falcon-rw-1.3B llm 0.80
models StableLM-3B-4e1t llm 0.80
datasets BoolQ llm 0.80
datasets MMLU llm 0.80
tasks LLM inference llm 0.80
tasks parameter-efficient fine-tuning llm 0.80
tasks LLM personalization llm 0.80
tasks natural language understanding llm 0.80
tasks common sense reasoning llm 0.80
hardware A100 80GB GPU llm 0.80
hardware flagship smartphone chipset llm 0.80
hardware 5G network (60/100 Mbps) llm 0.80
threat model data locality: cloud sees only activations and never raw user data or personal parameters (no cryptographic guarantee) llm 0.80
interaction other llm 0.80
code url https://github.com/ggerganov/llama.cpp (URL found in the full text) regex 0.50
code url https://github.com/PanQiWei/AutoGPTQ (URL found in the full text) regex 0.50
code url https://github.com/huggingface/peft (URL found in the full text) regex 0.50
limitations activation privacy is not protected; transmitted activations may leak information llm 0.80
limitations integrity of A, B, M matrices: unmatched parameters only slightly reduce performance rather than blocking access llm 0.80
limitations inference speed of PrivateLoRA is numerically estimated under assumed hardware/network specifications rather than fully measured end-to-end llm 0.80
limitations redundancy in adapting all decoder layers could be exploited to further cut communication llm 0.80
llm relevance high llm 0.80
key result communication reduction: >95% (low-rank residual transmission vs transmitting full hidden states) llm 0.80
key result throughput vs device-only: >300% (7B model under standard 5G network) llm 0.80
key result throughput vs single A100: >80% (33B model under standard 5G network) llm 0.80
key result tuning performance: comparable to LoRA (benchmarks incl. BoolQ and 5-shot MMLU) llm 0.80

System card: PrivateLoRA

Group 6. Trusted hardware, statistical and integrity-only alternatives
Family Split inference (edge-cloud)
Arithmetic Native
Interaction Interactive split
Model scale Llama-2-13B
Status Niche
Relation to the SoK Keeps private low-rank layers on device.
Note Activations are exposed to the cloud, so privacy is heuristic.
Systems page PrivateLoRA

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • library: TensorFlow 1 [p.3]
  • model: LLaMA 66 [p.3], LoRA 29 [p.1], Transformer 3 [p.3], GPT 1 [p.16], MLP 1 [p.5]
  • dataset: HellaSwag 4 [p.2]
  • hardware: GPU 27 [p.1], NVIDIA A100 15 [p.1], NVIDIA Jetson 1 [p.3], Raspberry Pi 1 [p.3], Multi-GPU 1 [p.12]
  • technique: Quantization 23 [p.1], Fine-tuning 11 [p.2], Federated learning 4 [p.1], Split learning / split inference 3 [p.1]
  • task: IoT / edge 41 [p.1], Language modeling / generation 28 [p.2]
  • topic: Large language models 50 [p.1], Benchmark 34 [p.2], Open source 4 [p.3]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/6-088_privatelora-for-efficient-privacy-preserving-llm.pdf catalogue 21 893.3 KB e8cb6d35c7c3

Database key: privatelora-for-efficient-privacy-preserving-llm · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set privatelora-for-efficient-privacy-preserving-llm COLUMN VALUE