> ## Documentation Index > Fetch the complete documentation index at: https://docs.corepillar.ai/llms.txt > Use this file to discover all available pages before exploring further. # Step 3: Results > Reading a Cost Calculator run: the recommendation and candidate cards, monthly scale, model comparison, spend history and forecast, cost and TCO breakdowns, Agent Audit, and per-scenario detail. The results page turns a run into a decision: which model to ship, what it will cost a month, and what to fix in the prompt. Every dollar on the page is a **measured** rate from this run, multiplied by a monthly volume you control. Nothing is invented; where a figure can't be measured, the page says so. The sections, top to bottom: 1. Recommendation and candidate cards 2. Best overall (when you compared two or more models) 3. Model Comparison 4. Historical Spend Breakdown and Spend Forecast 5. Cost Breakdown and TCO Breakdown 6. Assumptions & Formulas 7. Agent Audit 8. Detailed Scenario Results ## Recommendation The headline depends on the run: | Situation | Heading | | - | - | | Nothing executed with a valid key | "No recommendation yet" ("Add a model with a valid API key and run scenarios to get a recommendation.") | | One model | "*Model*'s performance for this project" | | Several models, and the best passes at least 80% of scenarios | "Recommendation for this Project — *Model*" | | Several models, none passes 80% | "No model passes reliably yet; closest: *Model*" | The winner ranks first on judged pass rate, then on mean judged quality. For a single model, the summary reads, for example: "*Model* passed 27/30 scenarios at 88% average accuracy, for $0.42 of real spend this run — $210/mo at 5K users." with "Add another model to see a cost/speed comparison." For several models, it summarizes the winner's judged passes and how it compares with the runner-up on cost, speed and accuracy. ### Candidate cards One card per model tested, each with a badge: | Badge | Meaning | | - | - | | **Right sized** | The recommended model | | **Viable** | Passes at least 80% of scenarios, not the pick | | **Marginal** | Passes 50% to 79%: "below the 80% needed to ship" | | **Fails task** | Passes under 50%: "fails most of the suite" | Hover the badge for "Passed *N*/*M* scenarios (judge-graded quality ≥ 0.70)". Each card shows **Judged quality**, **Cost / 1K tok** (cost per thousand tokens, so models are compared on price rather than on how much work they happened to do) and **Latency**. ### Headline tiles | One model | Several models | | - | - | | **Accuracy**: average accuracy across scenarios | **Cost Reduction** / **Cost Premium** / **Cost Difference**: the winner's cost per scenario against the baseline model ("\~0%" when within 3%) | | **Cost / mo at *scale***: measured cost per user times monthly users. The tooltip shows the arithmetic and what this run itself cost. | **Monthly Savings (*scale*)** or **Monthly Cost Increase (*scale*)**, from moving your volume from the baseline model to the winner | | **Latency**: average per scenario, or "—" if no span timings were recorded | **Judged quality**: the winner's mean, with the difference in points against the baseline | | **Passed**: *N*/*M* | **Speed**: how many milliseconds faster or slower the winner is on average | **Share Results** copies a plain-text summary to your clipboard: the project, the recommendation, pass count, accuracy, latency and cost, the full ranking, the monthly projection, the simulation ID, and a link that reopens this run (`?run=`). "Results summary + link copied". **Where latency comes from.** A scenario's latency is the sum of the durations of its spans: every model call and simulated tool call it made. It is not a separately stored timer. When a run has no span timings, latency tiles show "—" rather than zero. ## Best overall When you compared two or more models, a second card weighs the measured figures the way a person would, not by cost alone: * **AI pick**: a small Claude Haiku 4.5 call weighs accuracy, pass rate, latency and projected cost for up to eight candidates and writes a one-line rationale. It is funded from your organization's Platform Demo Key allowance, not your BYOK keys, and costs a fraction of a cent. * **Scored pick**: if that call is unavailable (no allowance, timeout, or an unusable answer), a deterministic score is used instead: accuracy 45%, pass rate 20%, cost 20% and speed 15%, with cost and speed scored relative to the best candidate. "The recommendation model was unavailable, so this is the deterministic best-overall pick: accuracy and pass rate weighed against cost and latency." An **Agrees with the ranking** badge appears when the pick matches the headline winner. ## Edit scale Every projected dollar on the page is a measured rate multiplied by a monthly volume. **Edit scale** opens the dialog that sets it. | Control | Details | | - | - | | **Monthly active users** | Default 5,000. Presets: 1K, 5K, 25K, 100K. Whole numbers of 1 or more. Changes preview live behind the dialog. | | **Advanced → Chats per user** | Default 3 per month. Choose per day, per month or per year; a day counts as 1/30 of a month and a year as 12 months. "An assumption about your traffic, not something this run measured." | | **Reset to 3 per month** | Appears once you change the chat rate. | The dialog shows "Projected monthly cost at *scale*" and, when relevant, "Projected monthly saving vs. costliest model" (several models) or "Projected monthly gap vs. Step 1 estimate" (one model). The Advanced section spells out the chain, for example "5,000 users × 3 chats = 15,000 requests/mo × $0.0004 = $6.00/mo". **Apply** keeps the change, **Cancel** puts everything back. The volume and chat rate are remembered for this project in your browser, and the Overview page's **Total Cost Analyzed** tile uses the same figure. ## Model Comparison Two tabs. **Model Performance**: a radar chart with four axes, "Accuracy (Answer correctness)", "Token Efficiency", "Overall Cost" and "Latency", a legend, and a table of each model's **Judged quality**, **Cost/scn**, **Latency** and **Pass rate**, with **Win** on the winner. **Scenario Performance**: every scenario, with one row per model, showing **Accuracy**, **Cost/1k** and **Latency** (click a header to sort). A model that failed the scenario is marked **Error**. Click a row to open the scenario. Long lists show the first scenarios with **See *N* more scenarios**. Category chips (happy path, edge case, multi turn, tool failure, refusal) filter the table. A category the run didn't generate is disabled: "This run generated no scenarios in this category." ## Historical Spend Breakdown Shown once the project has measured run spend. Every bar is the measured spend of Cost Calculator runs, by run date, starting at the first recorded run. * **Scope**: **Company** (all projects in the organization) or **Project**. * **Granularity**: daily, weekly, monthly, yearly. A granularity is disabled until there is enough history to fill it. * Stat boxes: **Measured spend**, **Budget** and **Variance**. No budget can be set today, so Budget reads "Not set" and Variance "—". * **Applications** (Company scope): the largest four projects by all-time measured spend, each with a "Next 3 mo" projection tagged **trend** (fitted to its own monthly history) or **flat** (held at its latest month). ## Spend Forecast Monthly spend projected forward from the current month, for the **Company** or **Project** scope. Stat boxes: **3 mo from now**, **This month**, **Starts**. Where there isn't enough history to fit a trend, the forecast assumes 8% month-over-month growth, and says so in the card description. ## Cost Breakdown How each scenario's cost splits between the agent's reasoning and each tool. Scopes: **Company** (the whole run rolled into one split), **Use Case** (the dominant scenario category) and **Project** (per scenario). Tool spans carry no token cost of their own, so their share is allocated by measured call count and latency. ## TCO Breakdown "Everything this project costs to run for a month at *scale*: measured model tokens, plus any cost lines you add." Add a line with a cost type, an optional **Custom name**, and a **\$ amount** per month, then click add. Cost types: Human-in-the-Loop (HITL), Cloud Infrastructure, Licensing & API Fees, Engineering & Maintenance, Data Pipeline & ETL, Monitoring & Observability, Security & Compliance, Training & Fine-tuning, Support & Operations, Vendor & Third-party SaaS. The donut shows **Total TCO** per month. Each line is tagged **measured** (model tokens from this run) or **custom** (your lines), with its share. Custom lines can be edited or removed. Nothing is assumed: if you add no lines, TCO equals measured model cost. Custom TCO lines live in the page while you have it open. They are not saved with the run. ## Assumptions & Formulas A collapsed panel under the TCO card with every parameter behind it and the arithmetic. **Volume basis** switches between **Per user** and **Per workflow**: per workflow lets you describe workflows (a primary and further ones) with a trigger frequency, instead of a user count. The panel shows **Estimated token usage**, **Total API cost** and **Total TCO**, and has a reset to the defaults. Changing the volume here changes every projection on the page. ## Agent Audit **Agent Audit — Failure Analysis** reads the run and proposes ranked system-prompt changes. * A chart of pass and fail by category, with a toggle between **This run** and **All runs** (every run in the project). * **Run Agent Audit** (when the run has failures) sends the failing transcripts and the error spans behind them to the model you tested and returns ranked fixes, most impactful first. When every scenario passed, the button is **Suggest improvements** and it proposes hardening for an already-working prompt instead. * Each fix shows the change as a diff and the scenarios behind it ("Failing cases behind this" or "Passing cases behind this"). Tick the fixes you want. * **Apply changes & rerun** (with the count of ticked fixes) applies the ticked fixes to the system prompt and returns you to [Step 1](/agenthub/cost-calculator/configure#after-applying-agent-audit-fixes), set up to re-test **the same scenarios**, so any change in the results comes from the prompt. A fix whose original text no longer appears in the prompt is left out and counted. * **Runs per case** (1 to 10) with a re-run button plays the cases again to accumulate failure data over several passes. The audit and the re-run passes use the model you are testing, on its key, and are billed to you. Each audit is one model call; each re-run pass executes the scenarios again. ## Detailed Scenario Results A collapsible list of every scenario, with a count (or "*N* measured · *N* projected" when synthetic rows exist). Each scenario shows: * Name, category badge, difficulty badge * **Pass**, **Partial** or **Fail** with the quality score. Hover for "Judged by GPT-4.1 (independent — never the model under test) against the scenario's goal and success criteria. Pass ≥ 0.70 quality; partial 0.50–0.69; critical failures fail outright." * The failure reason, and the judge's summary * Two bars: **Task + correctness** and **Instruction following** (or **Response match** and **Constraint compliance** when the judge was unavailable) * Measured cost * **View full scenario →** to open the conversation plan * For multi-model runs, one result line per model instead Tags you may see: | Tag | Meaning | | - | - | | **Offline plan** | The generator couldn't fill this slot, so a deterministic plan was used. It ran and cost real money. | | **Synthetic** | Generated from an imported run's trace profile and never executed. Its cost is a projection ("\~\$*X* proj."), and it is excluded from every pass rate, accuracy figure and measured total. | | **Not executed** | The scenario never ran, so there is no pass or fail. | The **Simulation ID** for the batch is shown with a copy button. Use it to find the batch's traces, which are tagged with it. ## Start over **Start over** returns to Step 1 with your configuration kept and the run cleared, ready for a new batch. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.