What a simulation run is
A simulation run is one Cost Calculator run for one model: a batch of scenarios plus, once executed, one result and one trace per scenario.The runs table
Runs are listed newest first, 20 at a time. Load more fetches the next page.
Click a row, or focus it and press Enter or Space, to open its detail panel beside the list. Click the row again, or press Escape, to close it. On a wide screen the list and the panel sit side by side with a draggable divider, and the Model and Created columns hide to make room.
Empty state: “No simulation runs yet — Run a cost analysis first — every scenario run there shows up here as a simulation run.” with Go to Cost Calculator.
Error state: if the list can’t load, an error message with a retry button.
Run detail panel
The panel header reads Simulation run, the first 12 characters of the ID, and a status. In the panel, a complete run is labelled Evaluated. Underneath: the model, the creation time, and the number of scenarios. Close hides the panel.Task success
The share of executed scenarios that passed, for example “90%” and “27 of 30 passed”, with a pass/fail bar and counts (”● 27 pass”, ”● 3 fail”). Only scenarios that actually ran are counted. When nothing has run yet: “No scenarios executed yet — run this simulation to see task success.”Grader categories
Average scores across the executed scenarios, from 0 to 100%. Bars are green at 80% or above, amber from 50%, red below.
A category with no scores is left out. “No grader scores yet — run this simulation to see category results.” when there are none at all.
The pass/fail here is each scenario’s recorded outcome. The Cost Calculator’s results page shows the richer judged verdict, including Partial, the quality score and the judge’s reason. Open the run there (from the Cost Calculator’s Recent strip, the Overview page, or
?run=<id> on the Cost Calculator URL) for the full picture.Simulation results
Tabs filter the scenario table: All N, Pass N, Fail N. A tab with a count of zero is disabled.
“No scenarios in this filter.” when a tab is empty.
Click a scenario row to open the scenario drawer: its conversation plan, its score profile, and Monitored parameters, the deterministic 0 to 1 scores for Tool Accuracy, Response Quality and Task Completion on this scenario.
Latency
The Simulations page doesn’t show latency. AgentHub does not store a separate latency figure for a simulated scenario; latency is the sum of the durations of the scenario’s spans (each model call and each simulated tool call). To see it:- open the scenario’s trace with View trace for the full span waterfall and timings, or
- open the run in the Cost Calculator, whose results show per-model and per-scenario latency computed from those spans.
Simulations and traces
Every executed scenario is a real trace withsource = simulation, tagged with its category, cost-calculator, the batch’s simulation ID and the model identifier. That makes them searchable in Traces and usable with evaluation collections like any production trace. Imported runs link to your original OpenTelemetry traces instead.
Permissions
Any AgentHub user in the organization can view Simulations. Starting runs needsrun_agents or manage_agents in the Cost Calculator. See Roles and permissions.