Legal Document Review System
Our role: Deployment & integration architect; technical direction of the build.
The Challenge
The client's legal team managed over 1,000 enterprise agreements. Every new contract or renewal required manual review to ensure all mandatory clauses were present and compliant — days per document, a bottleneck that delayed commercial deals. The team also had no efficient way to search the existing library; finding a precedent meant reading through files by hand.
Solution & Approach
The build was delivered by a vendor team strong in ML but new to production. We provided the cloud, deployment, and integration architecture, and directed the technical decisions that made it production-grade. The shipped system has two parts: a clause-by-clause reviewer that checks a draft agreement against a list of standard clauses and returns color-coded suggested revisions for a lawyer to approve, and a conversational assistant for querying the agreement library. We steered the team off a brittle custom-trained model onto a capable foundation model, simplified an over-engineered retrieval flow, and added deterministic tools so exact questions — counts, metadata — were answered reliably, backed by a real correctness regime of test sets, automated runs, and LLM-as-judge evaluation. A human lawyer reviews every result.
The hard call — simplify, and keep the reviewer in charge. A lawyer reviews everything regardless, so the win was never autonomy. We chose the simpler architecture — a capable model with deterministic tools over bespoke ML and heavy pre-processing — and spent the effort making the human’s review effortless: drop in a draft, get back color-coded revisions to approve.
- Clause-by-clause review — Checks a draft against a standard-clause list, one clause against one paragraph, and returns suggested revisions.
- Conversational agreement search — Query the library in plain English, with deterministic tools for exact counts and metadata.
- Secure by design — Enterprise IAM with Cognito authorization approved by the legal lead; confidential documents read and written only through a protected Lambda via service principal, inside a blessed SharePoint area.
- Human-in-the-loop — The system suggests; a lawyer decides, every time.
Outcome
In production use. By design, a lawyer reviews every result — the system's job was never to decide, but to make that review fast. A draft goes into a secured SharePoint folder and a revised one comes back in an adjacent folder, with changed clauses color-coded beneath the originals for instant comparison.
Facing a similar decision?
If the trade-offs look familiar, we can discuss whether our experience is relevant to your situation.
Start the conversation