Armature

Fetch one eval case in full: what the tester is instructed to do, the criteria it is judged against, its tool policy and limits, the suites it belongs to, and its most recent individual runs with outcome, score, duration, and any error. Use this to understand WHY a case failed before changing anything.

get_eval_case

Fetch one issue (problem cluster): its cause, in-range blast radius (sessions, tools, use cases, harnesses), and example sessions that hit it. Mirrors the Issues drawer.

get_issue

Fetch one reconstructed session end to end: header (including resolved user identity when recorded), the agent's intent, every event in start-time order with tool calls and search calls, classification, logs, and the latest judgement. Mirrors the session replay drawer.

get_session

Fetch a suite run: overall status (running / passed / failed), how many eval cases completed and passed, whether the result still matches the suite's current membership, and the per-case outcome with each underlying individual run. Omit suiteRunId to get the latest suite run for the suite — the answer to "is this MCP ready to ship right now".

get_suite_run

List the eval cases in one suite — exactly the set run_eval_suite dispatches. Each case carries its criteria count, schedule type, the other suites it belongs to, its last run outcome, and a pass rate over its recent runs. Runs the judge could not score are excluded from that rate, never counted as failures. Omit testSuiteId when the workspace has exactly one suite.

list_eval_cases

List the workspace's active eval suites, each with the MCP server it tests and the number of eval cases it would actually run. Start here when you need a testSuiteId for run_eval_suite or list_eval_cases. The case count reflects what "Run suite" dispatches, so an archived case is already excluded.

list_eval_suites

List the workspace's issues (recurring problem clusters) ranked by blast radius, with session KPIs (successful / failed / pass rate), daily pass-rate buckets, and each issue's cause category and affected use cases. Mirrors the Issues inbox.

list_issues

List active production MCP servers configured for the organization, including callable MCP targets and SDK analytics-only registrations. Each row's target_kind distinguishes mcp from analytics_ingest; auth-profile and latest connection-test metadata never include secret refs.

list_mcp_servers

List reconstructed agent sessions for the workspace, newest first, with each session's resolved user identity when recorded, primary use case, flag state, and latest judged outcome. Filter by outcome view, use case, MCP server, and time range. Mirrors the Sessions dashboard.

list_sessions

List the use-case clusters reconstructed from the workspace's sessions, each with volume, pass rate, and share of traffic. Before the classifier has promoted clusters, falls back to raw intent grouping (fallback: "raw_intents"). Mirrors the Use cases dashboard.

list_use_cases