Loop Engineering Workbench

Design, validate, and benchmark reliable AI agent loops.

A product workbench for developers who need to choose an architecture, bound execution, verify outcomes, estimate cost, export code, and improve loops across tools and models.

7 rules
MVP checks
9 formats
Exports
Success + cost + rounds
Benchmarks
10 official
Templates
Bounded Maker-Checker Loop
Max rounds 5 · Budget $5 · Checkpoint every round
Task Input
repo failing tests
Planner
gpt-5-mini · JSON plan schema
Executor
Codex · write sandbox · 25k tokens
Verifier
npm test · independent process
Pass -> Complete
Fail -> Retry -> Executor
Export targets
LangGraphOpenAI Agents SDKClaude CodeCodexPythonTypeScriptYAMLJSONMermaid
Template Registry

Start from reusable production loop templates.

Templates are not recipes. Each one carries a loop graph, config, prompt, schemas, permissions, supported tools, test results, average cost, rating, and repository link.

Test Repair
Bounded maker-checker
Avg cost
$1.92
Success
84%
ForkExportRun
PR Review
Reviewer with human gate
Avg cost
$0.46
Success
91%
ForkExportRun
Issue Triage
Classifier-router
Avg cost
$0.18
Success
88%
ForkExportRun
Dependency Upgrade
Plan-execute-verify
Avg cost
$1.27
Success
81%
ForkExportRun
Research
Research-synthesis loop
Avg cost
$0.74
Success
76%
ForkExportRun
Documentation Update
Diff-aware doc sync
Avg cost
$0.39
Success
93%
ForkExportRun
Security Review
Audit with approval
Avg cost
$1.64
Success
72%
ForkExportRun
Data Validation
Validator pipeline
Avg cost
$0.22
Success
95%
ForkExportRun
CI Repair
Failure recovery loop
Avg cost
$1.48
Success
83%
ForkExportRun
Release Preparation
Checklist gate
Avg cost
$0.58
Success
89%
ForkExportRun
01
Design
02
Check
03
Generate
04
Run
05
Observe
06
Compare
07
Improve

The platform is the workflow.

Documentation supports acquisition and onboarding, but the core product helps a developer finish a loop design and know whether it can run safely.

Loop Validator

Import a loop config, code snippet, or Mermaid graph and get a structured safety report before it runs.

Global stop conditionUnreachable nodesRetry boundsToken budgetTool permissions

Loop Simulator

Preview likely rounds, verifier outcomes, recovery points, and estimated spend without calling a model.

Round timelineNo-progress detectionCheckpoint planCost estimateFailure replay

Pattern Generator

Answer risk and verification questions, then generate a recommended agent-loop architecture.

Maker-checkerHuman gateParallel workersBounded retryRegression guard

Benchmark Database

Compare the same loops across Codex, Claude Code, LangGraph, and model providers with comparable metrics.

Success rateAverage costAverage roundsHuman interventionTime to pass
Loop Validator

Find structural, cost, permission, and quality risks before runtime.

The validator turns loop diagrams and configs into actionable engineering feedback: missing stops, impossible branches, unsafe tools, weak verifiers, unbounded spend, and recovery gaps.

Validation report
Critical
No global stop condition is defined.
Critical
Verifier and Executor share the same context, creating confirmation bias risk.
Warning
Retry path can loop forever after repeated verifier failures.
Warning
Token budget is missing for Planner and Executor nodes.
Suggestion
Trigger human review after two no-progress rounds.
Simulation: fix all failing tests
$5 budget
Round 1
Read failing tests, patch auth.ts, run verifier
23 passed / 4 failed
Round 2
Patch session.ts, restore checkpoint, run verifier
26 passed / 1 failed
Round 3
Same assertion repeats, no-progress policy fires
Human approval requested
Loop Simulator

See what the loop will do round by round.

Start with static execution simulation and cost estimation. Later, connect real model APIs to replay runs, compare recovery strategies, and inspect each tool call.

Benchmark Database

Compare loops by success rate and average cost.

Run the same task suite across agents, frameworks, and models. Track success rate, average cost, average rounds, human intervention rate, and time to pass.

Success rate
78%
Average cost
$2.10
Average rounds
4.8
Task typeProgramming languageRepository sizeModelToolLoop PatternVerification method
Runner
Task suite
Success
Avg cost
Avg rounds
Human
Codex
Repository repair
78%
$2.10
4.8 rounds
12%
Claude Code
Repository repair
82%
$2.90
4.1 rounds
18%
Cursor Agent
Repository repair
69%
$1.80
5.4 rounds
24%
LangGraph
Data QA workflow
91%
$0.68
2.6 rounds
7%
OpenAI Agents SDK
Tool approval flow
88%
$1.14
3.2 rounds
9%

MVP scope: designer, validator, and template gallery.

The first version can run entirely in the browser: add nodes, connect edges, configure retry and stop policies, save JSON, export Mermaid, share a link, and validate common risks.

Browser validation rules
Stop condition exists
Maximum loop count is set
Failure branch is defined
Verifier is present
Human approval is configured for risky actions
Cost or token budget is set
Deadlock loops are detected
Template can be forked, configured, exported, and versioned