Decision page
Private internal document assistant stack for Legal
What should a legal team use for a private assistant over internal documents?
ZBS editorial starting point. Start with Docling, pgvector, vLLM and LangGraph when controlled inference is a hard requirement.
Staff can search approved material without treating local hosting as proof of access control or quality.
Editorial starting point
Controlled internal retrieval
A concrete starting configuration that keeps source facts, model output and operational authority separate.
Choose this when: External model inference is prohibited and the firm can operate the stack.
-
Convert approved documents locally. The public quickstart documents local conversion and structured export.
Limit: Scans, unusual tables, annotations and source metadata require a representative parsing test.
Evidence: source 1
-
Search with access metadata. It keeps vector search beside relational metadata and access filters in PostgreSQL.
Limit: Approximate retrieval can miss results and needs recall measurement against an exact baseline.
Evidence: source 1
-
Serve the selected local model. It provides a documented self-operated model-serving layer.
Limit: Serving a model does not prove its task accuracy, safe tool use or secure operation.
Evidence: source 1
-
Coordinate retrieval and refusal. Stateful workflow steps keep model output separate from acceptance and operational action.
Limit: Policy, persistence, access control and recovery remain application responsibilities.
Evidence: source 1
Private / local
Controlled reasoning path
Keep parsing, retrieval or model inference in controlled infrastructure while retaining the same source-of-truth and approval rules.
Choose this when: Sensitive inputs cannot be sent to an external model API and the team can operate the additional infrastructure.
-
Ingestion
Docling
source backed inference
Use a separate sensitive-file profile. The public quickstart documents local conversion and structured export.
Limit: Scans, unusual tables, annotations and source metadata require a representative parsing test.
Evidence: source 1
-
Retrieval
pgvector
source backed inference
Keep exact and approximate search. It keeps vector search beside relational metadata and access filters in PostgreSQL.
Limit: Approximate retrieval can miss results and needs recall measurement against an exact baseline.
Evidence: source 1
-
Inference
vLLM
source backed inference
Serve an isolated smaller tier. It provides a documented self-operated model-serving layer.
Limit: Serving a model does not prove its task accuracy, safe tool use or secure operation.
Evidence: source 1
-
Route privileged requests. Stateful workflow steps keep model output separate from acceptance and operational action.
Limit: Policy, persistence, access control and recovery remain application responsibilities.
Evidence: source 1
Budget alternative
Lower-cost external model path
Keep the workflow and source integration explicit while evaluating a lower-cost model candidate on the same acceptance set.
Choose this when: External processing is acceptable and measured model spend is a leading constraint.
-
Ingestion
Docling
source backed inference
Convert documents locally. The public quickstart documents local conversion and structured export.
Limit: Scans, unusual tables, annotations and source metadata require a representative parsing test.
Evidence: source 1
-
Retrieval
pgvector
source backed inference
Select authorized source chunks. It keeps vector search beside relational metadata and access filters in PostgreSQL.
Limit: Approximate retrieval can miss results and needs recall measurement against an exact baseline.
Evidence: source 1
-
Provide an external cost comparison. It is a concrete lower-cost external model candidate for the same acceptance set.
Limit: Price alone is not task fitness; output structure, languages, availability and data terms need testing.
Evidence: source 1
-
Keep refusal rules explicit. Stateful workflow steps keep model output separate from acceptance and operational action.
Limit: Policy, persistence, access control and recovery remain application responsibilities.
Evidence: source 1
Trade-offs that change the choice
Implementation path
1. Define the boundary for files, chunks, prompts, logs and backups.
2. Carry permissions and deletion through the index.
3. Test cross-matter leakage and stale access.
4. Measure recall, grounding, exposure and operating cost.
Known limits
Private means a verified boundary, not merely local hosting.
The external budget path is incompatible when external inference is forbidden.
No product on this page is a universal winner; the configuration still needs a task-specific acceptance test.
EU and US routes stay consolidated with Global until evidence changes the answer.
Every component as data
The components in this stack are catalog entries. Fetch them with their versions, repository status and observation dates instead of reading this page.
GET /api/v1/catalog.json
