Capture → Orchestrate → Evaluate (COE) Checklist
Use this to systematize a single workflow. Keep it simple; iterate weekly.
Capture
- Define inputs, outputs, constraints
- Write a clear quality bar
- Collect 10 gold‑standard examples
- List tools/systems needed (read‑only first)
Orchestrate
- Chain steps with state (retrieve → generate → transform → format)
- Sanitize inputs; sign prompts; narrow permissions
- Add timeouts, rate limits, and budget caps
- Log inputs/prompts/outputs for traceability
Evaluate
- Scoring rubric (1–5) mapped to quality bar
- Sample 10–20 outputs/week
- Track success, intervention, and incident rates
- Gate high‑impact actions with human review