Skip to content

zkLLM: Zero Knowledge Proofs for Large Language Models

Authors: Haochen Sun; Jason Li; Hongyang Zhang
Year: 2024
Venue: CCS 2024

type: system collection: catalogue status: auto

Links: arXiv:2404.16109

Summary

Auto-extracted summary (unreviewed)

zkLLM is a zero-knowledge proof system for certifying that an LLM's output was genuinely produced by a given model while keeping the model parameters private, motivated by legal auditing of LLMs. It introduces tlookup, a parallelized lookup argument for non-arithmetic tensor operations with no asymptotic overhead, and zkAttn, a proof tailored to the attention mechanism that balances running time, memory and accuracy. With a fully parallelized CUDA implementation, a correctness proof for the entire inference of a 13-billion-parameter LLM is generated in under 15 minutes with a proof smaller than 200 kB.

Abstract

The recent surge in artificial intelligence (AI), characterized by the prominence of large language models (LLMs), has ushered in fundamental transformations across the globe. However, alongside these advancements, concerns surrounding the legitimacy of LLMs have grown, posing legal challenges to their extensive applications. Compounding these concerns, the parameters of LLMs are often treated as intellectual property, restricting direct investigations. In this study, we address a fundamental challenge within the realm of AI legislation: the need to establish the authenticity of outputs generated by LLMs. To tackle this issue, we present zkLLM, which stands as the inaugural specialized zero-knowledge proof tailored for LLMs to the best of our knowledge. Addressing the persistent challenge of non-arithmetic operations in deep learning, we introduce tlookup, a parallelized lookup argument designed for non-arithmetic tensor operations in deep learning, offering a solution with no asymptotic overhead. Furthermore, leveraging the foundation of tlookup, we introduce zkAttn, a specialized zero-knowledge proof crafted for the attention mechanism, carefully balancing considerations of running time, memory usage, and accuracy. Empowered by our fully parallelized CUDA implementation, zkLLM emerges as a significant stride towards achieving efficient zero-knowledge verifiable computations over LLMs. Remarkably, for LLMs boasting 13 billion parameters, our approach enables the generation of a correctness proof for the entire inference process in under 15 minutes. The resulting proof, compactly sized at less than 200 kB, is designed to uphold the privacy of the model parameters, ensuring no inadvertent information leakage.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution zkLLM, the first specialized zero-knowledge proof system for LLM inference llm 0.70
contribution tlookup, a parallelized lookup argument for non-arithmetic tensor operations in deep learning with no asymptotic overhead llm 0.70
contribution zkAttn, a zero-knowledge proof tailored to the attention mechanism llm 0.70
contribution fully parallelized CUDA implementation released as open source llm 0.70
contribution evaluation on open-source LLMs (OPT and a second family) up to 13B parameters llm 0.70
schemes zero-knowledge proofs llm 0.70
schemes lookup arguments llm 0.70
libraries CUDA llm 0.70
models OPT llm 0.70
models 13B-parameter LLM llm 0.70
tasks verifiable LLM inference llm 0.70
tasks zero-knowledge proof of attention llm 0.70
tasks LLM auditing llm 0.70
hardware GPU (CUDA, model not stated in excerpt) llm 0.70
threat model prover (model owner) convinces a verifier (auditor) that an inference is correct without revealing model parameters llm 0.70
interaction other llm 0.70
code url https://github.com/filecoin-project/ec-gpu (URL found in the full text) regex 0.50
code url https://github.com/herumi/mcl (URL found in the full text) regex 0.50
code url https://github.com/jvhs0706/zkllm-ccs2024 (URL found in the full text) regex 0.50
llm relevance medium llm 0.70
key result proof generation time: under 15 minutes (entire inference of a 13B-parameter LLM, CUDA implementation) llm 0.70
key result proof size: under 200 kB (13B-parameter LLM inference) llm 0.70

System card: zkLLM

Group 6. Trusted hardware, statistical and integrity-only alternatives
Family Zero-knowledge proofs
Arithmetic Field arithmetic
Interaction Non-interactive proof
Model scale OPT-13B
Status Active SOTA
Relation to the SoK Proves correct inference; does not hide the user's prompt from the server.
Note Complements CKKS: integrity rather than input confidentiality.
Code https://github.com/jvhs0706/zkllm-ccs2024
Systems page zkLLM

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • scheme: Zero-knowledge proofs 90 [p.1]
  • library: Orion 1 [p.15], PyTorch 1 [p.2]
  • model: CNN 9 [p.1], LLaMA 7 [p.3], Transformer 6 [p.4], GPT 5 [p.3], MLP 3 [p.1], SVM 2 [p.12], VGG 1 [p.12], Logistic regression 1 [p.12], Decision tree 1 [p.15]
  • hardware: CUDA 8 [p.1], GPU 3 [p.2], NVIDIA A100 1 [p.11], NVIDIA V100 1 [p.11], AMD EPYC 1 [p.11]
  • technique: Attention 20 [p.1], Quantization 20 [p.2], Softmax 18 [p.1], Lookup table 15 [p.2], Matrix multiplication 12 [p.2], Rescaling 7 [p.6], ReLU 7 [p.6], Semi-honest model 7 [p.5], Sigmoid 6 [p.8], Convolution 6 [p.4], GELU 4 [p.1], LayerNorm 4 [p.8], Non-interactive 4 [p.13], Fine-tuning 3 [p.8]
  • topic: Large language models 91 [p.1], Open source 5 [p.11]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/catalogue/6-089_zkllm-zero-knowledge-proofs-for-large-language-models.pdf catalogue 17 1.1 MB 39f2a2f29457

Database key: zkllm-zero-knowledge-proofs-for-large-language-models · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set zkllm-zero-knowledge-proofs-for-large-language-models COLUMN VALUE