Reports & Output

Run Directory

Every mcplab run creates a timestamped directory under mcplab/results/evaluation-runs (or --runs-dir if set). The directory contains four files.

• results.json — structured pass/fail for every scenario, assertion, and tool call.
• trace.jsonl — full tool call trace in newline-delimited JSON. One line per LLM exchange.
• report.html — self-contained interactive HTML report. Open it in a browser.
• summary.md — short markdown summary suitable for commit comments or Slack.

results.json

The results file records every scenario run: which agent was used, which tools were called, whether each assertion passed, and the final response. It is the source of truth for all downstream reports.

trace.jsonl

The trace captures the full sequence of LLM messages and tool calls for every scenario run. Each line is a JSON object representing one exchange. Use the trace to understand exactly what the agent did and why.

LLM-First Results Query Workflow

Use mcplab results commands when you need compact, structured output that is easy for LLMs and automation to consume.

mcplab results search auto-builds or refreshes local index when run artifacts changed. You can also prebuild manually with mcplab results index.

npx @inspectr/mcplab results list
npx @inspectr/mcplab results show --run <run-id> --format json

npx @inspectr/mcplab results search "tool failed timeout" --status failed --format json
npx @inspectr/mcplab results context --run <run-id> --scenario <scenario-id> --source trace --around 42 --format markdown

Regenerating a Report

The mcplab report command regenerates report.html from an existing run directory. Use this if you update the report template or if you want to re-render a report from a CI artifact.

npx @inspectr/mcplab report --input ./runs/2026-03-01T12-00-00

npx @inspectr/mcplab report --interactive