Web3 & EmergingLegal technology / AI2-week assessment plus retest
Securing a legal-tech LLM assistant before its enterprise launch
Series A legal-tech company, ~60 staff, LLM assistant over privileged documents
A legal-tech company was weeks from launching an LLM assistant that answers questions over clients' privileged documents — a product where one prompt injection could mean privileged material crossing a client boundary. An adversarial AI assessment proved the retrieval pipeline could be poisoned to exfiltrate context, drove architectural fixes, and left the team a red-team prompt suite to re-run on every model upgrade.
Representative engagement — the client is confidential, and the figures and the quoted comment illustrate typical outcomes for the work described.
- Injection paths proven pre-fix
- 5
- including document-borne indirect injection
- Cross-matter leakage after architectural fixes
- 0 reproduced
- on retest, against the same payloads
- Red-team prompt suite delivered
- 120+ payloads
- re-run on every model upgrade
- OWASP LLM Top 10 categories covered
- 10/10
- with per-category evidence
The challenge
- The assistant retrieved from privileged legal documents, so context leakage across client matters was an existential product risk.
- Enterprise law-firm prospects were asking security questions the team could not yet answer with evidence.
- Guardrails had been tuned against casual misuse, never against a motivated adversary using indirect injection.
What we did
- Attacked the assistant with direct, indirect and cross-user prompt injection, including payloads planted in uploaded documents the model would later retrieve.
- Attempted system-prompt extraction, retrieval-context exfiltration and abuse of every tool the agent could call.
- Reviewed the RAG pipeline's trust boundaries: embedding-store access controls, per-matter retrieval scoping and output handling.
- Ran a threat-modelling workshop with the engineering team so the fixes were owned, not just prescribed.
- Recommended architectural controls — retrieval scoping by matter, tool-call privilege boundaries, output filtering — and verified them on retest.
The outcome
- Document-borne indirect injection could no longer steer the assistant across matter boundaries after the retrieval-scoping fix, confirmed on retest.
- Findings mapped to the OWASP LLM Top 10 gave the sales team evidence-backed answers for enterprise security review.
- The client's team re-runs the delivered prompt suite on every model and prompt change as a regression gate.
“We thought our guardrails were solid because casual jailbreaks failed. They showed us what a motivated adversary does instead — and gave us the fixes that survive model upgrades.”