Skip to content

(a)i am not a lawyer, but...: Engaging legal experts towards responsible llm policies for legal advice

Authors: Inyoung Cheong; King Xia; K. J. Kevin Feng; Quan Ze Chen; Amy X. Zhang
Year: 2024
Venue: The 2024 ACM Conference on Fairness Accountability and Transparency

type: other collection: sok-references status: auto

Links: DOI · arXiv:2402.01864

Summary

Auto-extracted summary (unreviewed)

A FAccT 2024 human-computer-interaction and policy study that ran case-based workshops with 20 legal experts to work out when and why LLMs should or should not give legal advice. The authors distil the experts' deliberations into a four-dimension framework (user attributes and behaviours, nature of queries, AI capabilities, social impacts) and a set of response strategies that favour helping users find the right questions and relevant information over issuing legal opinions. The study surfaces considerations such as unauthorised practice of law, confidentiality and liability, and argues that case-based deliberation elicits more practice-informed policy guidance than surveys or abstract principles.

Abstract

Large language models (LLMs) are increasingly capable of providing users with advice in a wide range of professional domains, including legal advice. However, relying on LLMs for legal queries raises concerns due to the significant expertise required and the potential real-world consequences of the advice. To explore \textit{when} and \textit{why} LLMs should or should not provide advice to users, we conducted workshops with 20 legal experts using methods inspired by case-based reasoning. The provided realistic queries ("cases") allowed experts to examine granular, situation-specific concerns and overarching technical and legal constraints, producing a concrete set of contextual considerations for LLM developers. By synthesizing the factors that impacted LLM response appropriateness, we present a 4-dimension framework: (1) User attributes and behaviors, (2) Nature of queries, (3) AI capabilities, and (4) Social impacts. We share experts' recommendations for LLM response strategies, which center around helping users identify `right questions to ask' and relevant information rather than providing definitive legal judgments. Our findings reveal novel legal considerations, such as unauthorized practice of law, confidentiality, and liability for inaccurate advice, that have been overlooked in the literature. The case-based deliberation method enabled us to elicit fine-grained, practice-informed insights that surpass those from de-contextualized surveys or speculative principles. These findings underscore the applicability of our method for translating domain-specific professional knowledge and practices into policies that can guide LLM behavior in a more responsible direction.
Source: arxiv

Extracted details

Field Value Source Confidence
contribution case-based workshops with 20 legal experts on LLM legal advice llm 0.80
contribution four-dimension framework of factors affecting LLM response appropriateness llm 0.80
contribution expert recommendations for LLM response strategies (information vs opinion) llm 0.80
contribution identification of legal considerations such as unauthorised practice of law, confidentiality and liability llm 0.80
contribution case-based deliberation method for translating professional knowledge into LLM policies llm 0.80
tasks LLM legal advice llm 0.80
tasks responsible AI policy design llm 0.80
code url https://github.com/Social-Futures-Lab/case-law-ai-policy/blob/main/data/cases.csv.Workshop (URL found in the full text) regex 0.50
code url https://github.com/Social-Futures-Lab/case-law-ai-policy llm 0.80

SoK reference [2]

I. Cheong, K. Xia, K. J. K. Feng, Q. Z. Chen, and A. X. Zhang, “(a)i am not a lawyer, but...: Engaging legal experts towards responsible llm policies for legal advice,” in Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 2454–2469. [Online]. Available: https://doi.org/10.1145/3630106.3659048

Listed on the reference page of the site.

Keyword mentions

Counts of vocabulary terms in the full text (first page of occurrence in brackets).

  • model: GPT 6 [p.4], Transformer 1 [p.16], LLaMA 1 [p.16], Linear regression 1 [p.21]
  • technique: Fine-tuning 3 [p.3]
  • task: Medical / healthcare 19 [p.1], Finance 9 [p.1], Machine translation 2 [p.16]
  • topic: Large language models 158 [p.1], Benchmark 2 [p.16]

Files

Path (under papers/) Collection Pages Size SHA-256 Note
references/sok-references/002_a-i-am-not-a-lawyer-but-engaging-legal-experts-towards-respo.pdf sok-references 22 1.4 MB 39b7d4fa2047

Database key: a-i-am-not-a-lawyer-but-engaging-legal-experts-towards-responsible-llm · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set a-i-am-not-a-lawyer-but-engaging-legal-experts-towards-responsible-llm COLUMN VALUE