PrivateLoRA for Efficient Privacy Preserving LLM¶
Authors: Yiming Wang; Yu Lin; Xiaodong Zeng; Guannan Zhang
Year: 2023
Venue: arXiv 2023
type: system collection: catalogue status: auto
Links: arXiv:2311.14030
Summary¶
Auto-extracted summary (unreviewed)
PrivateLoRA proposes a heterogeneous LLM serving paradigm in which privacy-sensitive LoRA-style computation stays on the user's edge device while the shared decoder stack runs in the cloud, exchanging only activations. By exploiting the low rank of residual activations it cuts communication by over 95%, reaching more than 300% of device-only throughput for 7B models and over 80% of A100 throughput for 33B models under 5G, with tuning quality comparable to LoRA.
Abstract¶
End users face a choice between privacy and efficiency in current Large Language Model (LLM) service paradigms. In cloud-based paradigms, users are forced to compromise data locality for generation quality and processing speed. Conversely, edge device paradigms maintain data locality but fail to deliver satisfactory performance. In this work, we propose a novel LLM service paradigm that distributes privacy-sensitive computation on edge devices and shared computation in the cloud. Only activations are transmitted between the central cloud and edge devices to ensure data locality. Our core innovation, PrivateLoRA, addresses the challenging communication overhead by exploiting the low rank of residual activations, achieving over 95% communication reduction. Consequently, PrivateLoRA effectively maintains data locality and is extremely resource efficient. Under standard 5G networks, PrivateLoRA achieves throughput over 300% of device-only solutions for 7B models and over 80% of an A100 GPU for 33B models. PrivateLoRA also provides tuning performance comparable to LoRA for advanced personalization. Our approach democratizes access to state-of-the-art generative AI for edge devices, paving the way for more tailored LLM experiences for the general public. To our knowledge, our proposed framework is the first efficient and privacy-preserving LLM solution in the literature.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | device-cloud distributed LLM serving paradigm preserving data locality (only activations transmitted) | llm | 0.80 |
| contribution | PrivateLoRA: low-rank residual transmission reducing communication by over 95% | llm | 0.80 |
| contribution | throughput analysis showing >300% of device-only speed for 7B and >80% of A100 for 33B models under 5G | llm | 0.80 |
| contribution | tuning performance comparable to LoRA for personalization | llm | 0.80 |
| libraries | LM-Eval Harness | llm | 0.80 |
| libraries | GPTQ | llm | 0.80 |
| libraries | AutoGPTQ | llm | 0.80 |
| libraries | llama.cpp | llm | 0.80 |
| libraries | PEFT | llm | 0.80 |
| models | LLaMA-2-7B | llm | 0.80 |
| models | LLaMA-2-13B | llm | 0.80 |
| models | 33B LLaMA model | llm | 0.80 |
| models | OPT-1.3B | llm | 0.80 |
| models | Falcon-rw-1.3B | llm | 0.80 |
| models | StableLM-3B-4e1t | llm | 0.80 |
| datasets | BoolQ | llm | 0.80 |
| datasets | MMLU | llm | 0.80 |
| tasks | LLM inference | llm | 0.80 |
| tasks | parameter-efficient fine-tuning | llm | 0.80 |
| tasks | LLM personalization | llm | 0.80 |
| tasks | natural language understanding | llm | 0.80 |
| tasks | common sense reasoning | llm | 0.80 |
| hardware | A100 80GB GPU | llm | 0.80 |
| hardware | flagship smartphone chipset | llm | 0.80 |
| hardware | 5G network (60/100 Mbps) | llm | 0.80 |
| threat model | data locality: cloud sees only activations and never raw user data or personal parameters (no cryptographic guarantee) | llm | 0.80 |
| interaction | other | llm | 0.80 |
| code url | https://github.com/ggerganov/llama.cpp (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/PanQiWei/AutoGPTQ (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/huggingface/peft (URL found in the full text) | regex | 0.50 |
| limitations | activation privacy is not protected; transmitted activations may leak information | llm | 0.80 |
| limitations | integrity of A, B, M matrices: unmatched parameters only slightly reduce performance rather than blocking access | llm | 0.80 |
| limitations | inference speed of PrivateLoRA is numerically estimated under assumed hardware/network specifications rather than fully measured end-to-end | llm | 0.80 |
| limitations | redundancy in adapting all decoder layers could be exploited to further cut communication | llm | 0.80 |
| llm relevance | high | llm | 0.80 |
| key result | communication reduction: >95% (low-rank residual transmission vs transmitting full hidden states) | llm | 0.80 |
| key result | throughput vs device-only: >300% (7B model under standard 5G network) | llm | 0.80 |
| key result | throughput vs single A100: >80% (33B model under standard 5G network) | llm | 0.80 |
| key result | tuning performance: comparable to LoRA (benchmarks incl. BoolQ and 5-shot MMLU) | llm | 0.80 |
System card: PrivateLoRA¶
| Group | 6. Trusted hardware, statistical and integrity-only alternatives |
| Family | Split inference (edge-cloud) |
| Arithmetic | Native |
| Interaction | Interactive split |
| Model scale | Llama-2-13B |
| Status | Niche |
| Relation to the SoK | Keeps private low-rank layers on device. |
| Note | Activations are exposed to the cloud, so privacy is heuristic. |
| Systems page | PrivateLoRA |
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- library: TensorFlow 1 [p.3]
- model: LLaMA 66 [p.3], LoRA 29 [p.1], Transformer 3 [p.3], GPT 1 [p.16], MLP 1 [p.5]
- dataset: HellaSwag 4 [p.2]
- hardware: GPU 27 [p.1], NVIDIA A100 15 [p.1], NVIDIA Jetson 1 [p.3], Raspberry Pi 1 [p.3], Multi-GPU 1 [p.12]
- technique: Quantization 23 [p.1], Fine-tuning 11 [p.2], Federated learning 4 [p.1], Split learning / split inference 3 [p.1]
- task: IoT / edge 41 [p.1], Language modeling / generation 28 [p.2]
- topic: Large language models 50 [p.1], Benchmark 34 [p.2], Open source 4 [p.3]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/catalogue/6-088_privatelora-for-efficient-privacy-preserving-llm.pdf |
catalogue | 21 | 893.3 KB | e8cb6d35c7c3 |
Database key: privatelora-for-efficient-privacy-preserving-llm · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set privatelora-for-efficient-privacy-preserving-llm COLUMN VALUE