Example Output: RAG Knowledge Ops Executive Decision Memo
Inputs used
- Project context: a support answer engine grounded in product docs and release notes
- Target audience: AI engineers, knowledge managers, legal ops, support ops
- Success metric: activation, quality, and risk reduction
- Available tools and data: vector database, document parser, reranker, eval set, access control list
- Desired depth: Production-ready
- Output tone: Clear operator memo
Generated Result
one-page recommendation, options table, risks, and next actions
Decision needed
The immediate decision is whether a support answer engine grounded in product docs and release notes is mature enough for a controlled pilot. The strongest evidence should come from source documents and chunk samples; if either source is missing, mark the recommendation as provisional rather than filling the gap with assumptions.
Recommendation
Recommendation: run a narrow pilot before broad rollout. Prefer a governance-forward pilot if evidence suggests citation mismatch; prefer a speed-forward pilot only when chunk samples and failed queries are already reliable.
Options
Option A optimizes speed by shipping a limited workflow around failed queries. Option B optimizes control by adding reviewer sign-off and rollback steps. Option C waits until evidence from citation audits is stronger. Use the same success metric for all three options.
Evidence
Evidence to trust: citation audits, source documents, and reviewer notes from eval set. Evidence to treat cautiously: anecdotes that are not tied to a time window, cohort, or source owner.
Risks
Treat stale retrieval as a launch blocker until there is a control that can be verified. The minimum control is: freshness policy, plus reviewer sign-off for ambiguous outputs.
Next actions
Next actions: validate chunk samples, assign a reviewer for citation mismatch, and schedule a decision checkpoint after the first pilot cohort. Do not expand scope until the review path works in practice.
Recommended Decision
Proceed with a narrow pilot focused on source documents and chunk samples. Treat stale retrieval as the primary launch blocker. The first milestone should prove that the workflow produces a usable RAG blueprint, source policy, and retrieval eval plan with clear evidence, named owners, and a review path for ambiguous cases.
Expected quality checks
- The result is specific to RAG ingestion, retrieval quality, citation design, and knowledge base governance.
- It includes the required sections: Decision needed, Recommendation, Options, Evidence, Risks, Next actions.
- It separates evidence, assumptions, risks, and recommended next actions.
- It includes practical verification steps, not only generic advice.
- It names the most important failure mode for this domain: stale retrieval.
Reuse note
Before copying the output into production work, replace all default variables with your real data and run a human review for high-impact decisions.