Example Output: Audio Voice Evaluation and Red-Team Harness
Inputs used
- Project context: a realtime voice support agent for onboarding new SaaS customers
- Target audience: voice UX teams, support ops, media producers, accessibility teams
- Success metric: activation, quality, and risk reduction
- Available tools and data: call analytics, voice model, conversation simulator, QA rubric
- Desired depth: Production-ready
- Output tone: Clear operator memo
Generated Result
eval matrix, adversarial cases, grading rubric, and release threshold
Success criteria
Create at least 12 golden tasks: 6 normal cases, 3 edge cases, and 3 adversarial cases targeting uncanny tone. A passing result must cite the evidence source and state confidence.
Golden tasks
Create at least 12 golden tasks: 6 normal cases, 3 edge cases, and 3 adversarial cases targeting long turns. A passing result must cite the evidence source and state confidence.
Adversarial tasks
Use fallback logs as evidence, apply the constraint "accessible language", and explicitly note how the plan reduces unsafe escalation. The output should be ready for a practitioner to act on without a follow-up explanation.
Rubric
Use latency metrics as evidence, apply the constraint "explicit handoff", and explicitly note how the plan reduces misheard user intent. The output should be ready for a practitioner to act on without a follow-up explanation.
Sampling plan
Release in three gates: internal dry run, limited pilot, then measured expansion. Each gate must show evidence that low-latency phrasing is true in practice, not only in documentation.
Release decision
Release in three gates: internal dry run, limited pilot, then measured expansion. Each gate must show evidence that clear repair paths is true in practice, not only in documentation.
Recommended Decision
Proceed with a narrow pilot focused on call transcripts and voice personas. Treat uncanny tone as the primary launch blocker. The first milestone should prove that the workflow produces a usable conversation script, fallback matrix, and voice QA checklist with clear evidence, named owners, and a review path for ambiguous cases.
Expected quality checks
- The result is specific to AI-assisted voice agents, realtime audio UX, script writing, and conversation QA.
- It includes the required sections: Success criteria, Golden tasks, Adversarial tasks, Rubric, Sampling plan, Release decision.
- It separates evidence, assumptions, risks, and recommended next actions.
- It includes practical verification steps, not only generic advice.
- It names the most important failure mode for this domain: uncanny tone.
Reuse note
Before copying the output into production work, replace all default variables with your real data and run a human review for high-impact decisions.