Most product teams don’t need to build a model. They need to decide which models to integrate, how to evaluate them honestly, what governance controls to put in place before the security team asks, and which parts of the AI stack are actually their problem versus a vendor’s problem.
This page covers all of that — the stack layers that matter, how to evaluate before you’re in production trouble, and the security patterns that apply specifically to sensitive document and workflow contexts.
The AI/ML software stack: what’s actually your problem
Modern AI products have multiple layers. Which ones you own determines your risk and your flexibility.
Layer 1: Foundation models (usually vendor-managed)
OpenAI, Anthropic, Google Gemini, and open-weight alternatives (Mistral, Llama) provide the inference layer. For most product teams, this is an API call, not something you run. The decision here is provider selection based on capability, cost, compliance posture, and rate limit terms — not architecture.
Layer 2: Orchestration and memory (where most product work happens)
This is where product teams spend most of their engineering effort: retrieval pipelines, prompt management, tool calling, conversation state, and output parsing. Tools here include LangChain, LlamaIndex, and increasingly bespoke implementations as teams learn where frameworks add friction. Evaluate whether you need a framework or just clean abstractions over the raw API.
Layer 3: Data infrastructure
What your AI can access determines what it can do. Vector databases (Pinecone, Weaviate, pgvector) for semantic retrieval, feature stores for structured signal, and object storage for document access. For document-heavy workflows, how you chunk, index, and retrieve directly affects output quality — this is an engineering problem, not a model problem.
Layer 4: Evaluation and observability
The layer most teams underinvest in until something breaks in production. You need: offline evaluation against a labelled set before you ship, production tracing that shows you exactly what prompt, retrieval context, and output was involved in each response, and monitoring for quality regression over time. LangSmith, Helicone, Braintrust, and custom implementations all serve this role.
Layer 5: Guardrails
Output filtering, PII detection, prompt injection resistance, and refusal policies. Not optional for any product handling sensitive content. Can be implemented as a layer over any model via services like Guardrails AI or LlamaGuard, or as prompt-level rules.
How to evaluate AI features before production
The most common failure pattern: a promising prototype gets shipped with minimal evaluation because it “looked good in demos.” The structured evaluation approach that avoids this:
1. Define the task precisely
“Summarise documents” is not a task definition. “Extract the key risk factors from a legal agreement and present them in priority order for a non-legal reader” is. Precision here drives metric selection.
2. Build a representative test set
Minimum 50–100 examples per task, including edge cases and adversarial inputs. For document workflows: mixed quality documents, ambiguous inputs, documents with conflicting information.
3. Choose metrics that match the use case
- Factual accuracy: does the output correctly reflect source material?
- Citation quality: are claims grounded in referenced documents?
- Refusal rate: does the system correctly decline unanswerable queries?
- Latency and cost per task: production economics, not just quality
4. Run offline before going live
Evaluate your candidate prompt + model combination against your test set. Iterate until you have a quality baseline you can measure regression against.
5. Monitor in production
Set alerts for latency spikes, cost anomalies, and customer-reported quality failures. Review a sample of production outputs weekly.
Governance patterns for sensitive AI workflows
For AI features that touch confidential documents — financial data, legal materials, due diligence files — the governance requirements mirror VDR-style access control:
Least privilege retrieval: the retrieval layer should only surface documents the requesting user is authorised to access. This is an access control problem, not a model problem. Implement it in the retrieval step, not via prompt instructions.
Prompt and output logging: every AI interaction that involves sensitive material should be logged with sufficient context to reconstruct what happened. This is a compliance and incident response requirement.
PII and secrets handling: never pass unredacted PII or credentials into external model APIs unless your compliance posture explicitly allows it. Build a preprocessing step.
Human review for high-stakes outputs: for AI outputs that support financial decisions, legal interpretations, or investor communications, require a human sign-off step before the output is acted upon. AI assists the decision; a human owns it.
What’s actually worth your engineering time in 2026
Given the pace of model improvement, the right investment hierarchy for most product teams:
- Evaluation infrastructure — this doesn’t go to waste when models improve; it tells you when a new model is better for your task
- Retrieval quality — chunking strategy, indexing approach, and reranking have a larger effect on output quality than model selection in most RAG applications
- Observability tooling — production visibility compounds over time
- Model selection — evaluate against your task, not general benchmarks
Fine-tuning and custom training are rarely the right first investment. Start with retrieval and evaluation.
FAQ
When you have stable, high-quality labelled examples (500+), a clear task definition that off-the-shelf models handle inconsistently, and latency or cost requirements that make hosted API calls impractical. For most teams, this is not the first year of AI feature development.
Model providers update models; your data distribution changes; user behaviour evolves. Set up production evaluation sampling and alert when your quality metrics drop below defined thresholds. Without monitoring, degradation is invisible.
Insufficient retrieval access control — building a RAG system where the retrieval layer doesn’t respect document-level permissions, allowing one user’s query to surface documents they shouldn’t access. Fix this at the retrieval layer before any other security work.