(a)i am not a lawyer, but...: Engaging legal experts towards responsible llm policies for legal advice¶
Authors: Inyoung Cheong; King Xia; K. J. Kevin Feng; Quan Ze Chen; Amy X. Zhang
Year: 2024
Venue: The 2024 ACM Conference on Fairness Accountability and Transparency
type: other collection: sok-references status: auto
Links: DOI · arXiv:2402.01864
Summary¶
Auto-extracted summary (unreviewed)
A FAccT 2024 human-computer-interaction and policy study that ran case-based workshops with 20 legal experts to work out when and why LLMs should or should not give legal advice. The authors distil the experts' deliberations into a four-dimension framework (user attributes and behaviours, nature of queries, AI capabilities, social impacts) and a set of response strategies that favour helping users find the right questions and relevant information over issuing legal opinions. The study surfaces considerations such as unauthorised practice of law, confidentiality and liability, and argues that case-based deliberation elicits more practice-informed policy guidance than surveys or abstract principles.
Abstract¶
Large language models (LLMs) are increasingly capable of providing users with advice in a wide range of professional domains, including legal advice. However, relying on LLMs for legal queries raises concerns due to the significant expertise required and the potential real-world consequences of the advice. To explore \textit{when} and \textit{why} LLMs should or should not provide advice to users, we conducted workshops with 20 legal experts using methods inspired by case-based reasoning. The provided realistic queries ("cases") allowed experts to examine granular, situation-specific concerns and overarching technical and legal constraints, producing a concrete set of contextual considerations for LLM developers. By synthesizing the factors that impacted LLM response appropriateness, we present a 4-dimension framework: (1) User attributes and behaviors, (2) Nature of queries, (3) AI capabilities, and (4) Social impacts. We share experts' recommendations for LLM response strategies, which center around helping users identify `right questions to ask' and relevant information rather than providing definitive legal judgments. Our findings reveal novel legal considerations, such as unauthorized practice of law, confidentiality, and liability for inaccurate advice, that have been overlooked in the literature. The case-based deliberation method enabled us to elicit fine-grained, practice-informed insights that surpass those from de-contextualized surveys or speculative principles. These findings underscore the applicability of our method for translating domain-specific professional knowledge and practices into policies that can guide LLM behavior in a more responsible direction.
Source: arxiv
Extracted details¶
| Field | Value | Source | Confidence |
|---|---|---|---|
| contribution | case-based workshops with 20 legal experts on LLM legal advice | llm | 0.80 |
| contribution | four-dimension framework of factors affecting LLM response appropriateness | llm | 0.80 |
| contribution | expert recommendations for LLM response strategies (information vs opinion) | llm | 0.80 |
| contribution | identification of legal considerations such as unauthorised practice of law, confidentiality and liability | llm | 0.80 |
| contribution | case-based deliberation method for translating professional knowledge into LLM policies | llm | 0.80 |
| tasks | LLM legal advice | llm | 0.80 |
| tasks | responsible AI policy design | llm | 0.80 |
| code url | https://github.com/Social-Futures-Lab/case-law-ai-policy/blob/main/data/cases.csv.Workshop (URL found in the full text) | regex | 0.50 |
| code url | https://github.com/Social-Futures-Lab/case-law-ai-policy | llm | 0.80 |
SoK reference [2]¶
I. Cheong, K. Xia, K. J. K. Feng, Q. Z. Chen, and A. X. Zhang, “(a)i am not a lawyer, but...: Engaging legal experts towards responsible llm policies for legal advice,” in Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 2454–2469. [Online]. Available: https://doi.org/10.1145/3630106.3659048
Listed on the reference page of the site.
Keyword mentions¶
Counts of vocabulary terms in the full text (first page of occurrence in brackets).
- model: GPT 6 [p.4], Transformer 1 [p.16], LLaMA 1 [p.16], Linear regression 1 [p.21]
- technique: Fine-tuning 3 [p.3]
- task: Medical / healthcare 19 [p.1], Finance 9 [p.1], Machine translation 2 [p.16]
- topic: Large language models 158 [p.1], Benchmark 2 [p.16]
Files¶
| Path (under papers/) | Collection | Pages | Size | SHA-256 | Note |
|---|---|---|---|---|---|
references/sok-references/002_a-i-am-not-a-lawyer-but-engaging-legal-experts-towards-respo.pdf |
sok-references | 22 | 1.4 MB | 39b7d4fa2047 |
Database key: a-i-am-not-a-lawyer-but-engaging-legal-experts-towards-responsible-llm · created 2026-11-05 · updated 2026-11-05 · edit with python3 tools/paperdb.py set a-i-am-not-a-lawyer-but-engaging-legal-experts-towards-responsible-llm COLUMN VALUE