Robust Multi-Stage LLM Agent Workflow with External Deterministic Gates for Verifiable Outcomes

Workflow value: 90/100
Status: active · Freshness: 70/100 · Confidence: 0.95 · Level: advanced
Categories: Quality Control, Token Saving, Context & Memory, Debugging, Shipping, Multi-Agent
Original source: r/ClaudeAI post/comment

What problem this solves

Building robust, verifiable, and reliable multi-stage LLM agent workflows by strictly separating agent generation from deterministic verification, thereby preventing agents from 'hallucinating' success and improving overall system integrity.

Summary

A six-stage LLM agent workflow (research, spec, decompose, implement, review, ship) where stages communicate exclusively via files on disk. Crucially, verification is handled by external, deterministic 'gates' (plain shell/python scripts) that check the artifacts produced by the agent. This ensures the agent never grades its own work, leading to more reliable outcomes, easier debugging, and the ability to use cheaper models by offloading objective checks.

Why it is useful

This workflow provides a highly valuable and robust framework for building reliable LLM agent systems. Its core innovation lies in the strict separation of agent generation from deterministic verification, ensuring that LLMs do not 'grade their own homework.' This approach significantly enhances the trustworthiness and reliability of LLM-generated outputs, reduces debugging effort, and allows for cost optimization by leveraging cheaper models for generation while relying on precise, token-free external tools for validation. It's a foundational pattern for anyone seeking to build production-ready, verifiable LLM-powered applications.

Workflow

1. Define a multi-stage workflow (e.g., research, spec, decompose, implement, review, ship) where each stage has a clear input and output.
2. Design each stage to communicate only through files on disk, using specific output directories and naming conventions (e.g., .research-output/{slug}.md).
3. Configure each stage using a config object that specifies its ID, prompt file, model, allowed tools (e.g., bash, edit), verification gates, iteration limits, cost limits, and environment variables.
4. Implement 'gates' as external, deterministic shell/python scripts for each stage. These gates are responsible for verifying the artifacts produced by the agent.
5. Ensure gate scripts exit with 0 for pass, 2 for fail, or any other code for an infrastructure error.
6. Execute the agent for a specific stage.
7. Immediately after the agent exits, run that stage's configured gates.
8. If all gates pass, the stage is considered complete. If any gate fails, its stderr becomes the next prompt for the agent (or the overall process fails if limits are hit).
9. Implement specific gate checks: for all stages (required sections, artifact freshness, no secrets), research (claims trace to sources), spec (claims re-checked against cited artifacts), decompose (issues exist in tracker), implement (tsc --noEmit, full test suite, non-stub evidence), and ship (review verdict matches HEAD's sha, not expired).

Tools / artifacts

• .research-output/{slug}.md
• .spec-output/{slug}.md
• .issues-output/{ISSUE}.md
• .impl-output/{ISSUE}.md
• .ship-output/verdict.json
• Stage configuration object (JSON/YAML)
• Prompt files (e.g., prompts/implement.md)
• Bash/Shell scripts (for gates)
• Python scripts (for gates)
• Tools like bash, edit, find, grep, ls, read, write
• TypeScript compiler (tsc) for type checking

Validation signals

• Explicit statement: 'The agent never grades its own work.'
• Explicit statement: 'Every gate answers one falsifiable question about what's actually on disk, and it can't be argued with.'
• Anecdote: LLM reviewer performance improved by offloading deterministic checks (compile, tests pass) to gates.
• Benefit: Allows using cheaper models because they only need to be good enough to pass a gate, not trusted unsupervised.
• Benefit: Deterministic tools like tsc cost zero tokens and are never wrong, making them superior to LLM opinions for objective checks.

Cautions

• Includes a 'no-secrets' gate that redacts secrets in place before failing, enhancing security.

Limitations

• The post describes the architecture but does not provide concrete code examples for the orchestrator or gate scripts, requiring users to implement them from scratch.
• Initial setup complexity might be high for users unfamiliar with scripting, CI/CD principles, or agent orchestration.
• The specific gate checks are tailored to a coding project; users would need to adapt them for other domains or project types.

Rate this workflow

Upvote this post if the workflow is useful, reproducible, or worth recommending.

Downvote if it is vague, outdated, unsafe, overhyped, or not reproducible.

Reply if it worked for you, failed, is outdated, or has a better alternative.

This post was generated automatically from the workflow library database.