Openlayer

Trigger a one-off evaluation of a monitoring test over a time range.

evaluate_test

Fetch rows (traces / inference data points) from a data source. Use this to inspect the structure and contents of production traces -- e.g. to understand which columns exist before proposing tests. Rows come back untrimmed, so they include each test's judge score (``openlayer_metric_score_*`` / ``openlayer_llm_rubric_score_*``) alongside the judge's explanation in the matching ``*_explanation_*`` column. To go the other way -- from a failing test to the rows that caused it -- use ``fetch_failed_rows_for_goal``.

fetch_data_source_rows

Fetch session-level aggregations for a data source. Sessions group related traces/records; use this to analyze conversation or request sessions before proposing session-level tests.

fetch_data_source_sessions

Fetch the rows that made a test fail, with per-row judge scores and the LLM judge's explanation (reasoning) for each score. This is how you answer "why did this test fail on this row?" -- each returned row carries a ``judge_scores`` list of ``{score_column, score, test_name, explanation}``, where ``explanation`` is the judge's written justification, the same text the UI shows in the record detail panel. Rows are selected using a 3-tier filter strategy: 1) Prefer pre-computed rowsBody.columnFilters from the goal result (update measurement names with llmSettingHash if available; invert operators to target failed rows). 2) Otherwise construct aggregate metric filters (latency/cost/tokens).

fetch_failed_rows_for_goal

Create a commit from a bundle already uploaded via create_presigned_upload_url.

finalize_commit

Check whether a data source is still receiving data, and how much. Returns when the last sample arrived plus request-volume series. Use it first when a user reports missing data or asks whether an integration is still working: a stale ``dateLastSampleReceived`` says the problem is upstream of Openlayer, not in their tests. The volume series come in fixed hourly/daily/weekly/monthly windows computed by the backend -- the route accepts no time range, so an arbitrary window (say, one specific day last week) cannot be requested here. Use ``fetch_data_source_rows`` with timestamp filters for that.

get_data_source_health

Fetch one row (trace) from a data source by inference ID, with its OTel steps. Direct lookup for when you already have an inference ID -- from a user, a log, or a previous search -- so you don't have to build a filter to find one trace.

get_data_source_row

Get the aggregation for one session in a data source. The drill-down for ``fetch_data_source_sessions``: returns the session's record count, token usage, cost, duration and scores.

get_data_source_session

Read the cached AI summary of a session, plus a staleness flag. A whole conversation compressed into a paragraph -- far cheaper to reason over than paging the raw turns. Read-only: it never triggers generation. Summary generation is gated by workspace AI-feature settings and is deliberately not exposed as a tool, so a session with no summary yet reports that explicitly rather than looking like an empty summary. ``stale`` is true when the session has changed since the summary was written.

get_data_source_session_summary

Get the aggregation for one user of a data source. Use when you already have a user ID and want that user's session count, record count, token usage and cost without paging the whole cohort.

get_data_source_user

Get per-project rule pass/fail stats for a framework.

get_framework_project_rule_stats

Get the evidence attached to a rule result.

get_rule_result_evidence

List results for a test.

get_test_results

Get aggregate rule-result stats for a workspace (compliance summary).

get_workspace_rule_stats

List the test results for a commit. Each result is shaped to the essentials: test_name, status, threshold, the measured value, and goal_id -- pass that goal_id to ``fetch_failed_rows_for_goal`` to drill into the failing rows and read the LLM judge's per-row scores and explanations.

list_commit_test_results

List the commits in a project.

list_commits

List one user's sessions in a data source. The cohort investigation path: ``list_data_source_users`` surfaces a user whose cost or latency stands out, then this shows that user's sessions.

list_data_source_user_sessions

List per-user activity aggregations for a data source. Returns users who interacted with the data source along with session counts, record counts, token usage, and costs -- useful for cohort/session analysis.

list_data_source_users

List the documents (e.g. the underlying standard/policy) for a framework.

list_framework_documents

List the rules that belong to a framework.

list_framework_rules

List the compliance/governance frameworks in a workspace.

list_frameworks

List the test results for an inference pipeline (data source). Each result is shaped to the essentials: test_name, status, threshold, the measured value, and goal_id -- pass that goal_id to ``fetch_failed_rows_for_goal`` to drill into the failing rows and read the LLM judge's per-row scores and explanations.

list_inference_pipeline_test_results

List the inference pipelines in a project.

list_inference_pipelines

Get a list of the projects in the user's workspace.

list_projects

List the per-project results for a rule.

list_rule_results

List the rule tags defined in a workspace. Tags categorize rules (e.g. by topic or control family); use them when composing a framework or filtering rules.

list_rule_tags

List the governance rules in a workspace.

list_rules

List the sessions behind a session-level test result, with their scores. Use this to answer "why did this session test fail?". Each session comes back enriched with the test's session-metric score, its explanation, and pass/fail status. This is the session-level counterpart to ``fetch_failed_rows_for_goal``, which only reaches row-level failures. A ``null`` score does not necessarily mean the session was unscored -- it depends on the data source's backend. ClickHouse-native data sources return only sessions that have a recorded score in the result's evaluation window; other backends return every session in the window, with ``score`` null where the metric had no resolved value.

list_test_result_sessions

List the tests defined in a project. Returns a summary per test (name, type, subtype, a one-line threshold, dataset usage). The full threshold/rubric is intentionally omitted here to save tokens.

list_tests

List rule results across a workspace (the compliance posture).

list_workspace_rule_results

List the workspaces you belong to, and which one this connection uses. This connection can only act on its own workspace. To work in another one, add a second Openlayer MCP connection and pick that workspace when authorizing it.

list_workspaces

Retrieve a commit by ID.

retrieve_commit

Retrieve a framework by ID.

retrieve_framework

Retrieve a framework document's content. The full document can be large (tens of thousands of tokens). Set ``sections_only=True`` to get just the section titles (the outline), then fetch the full content only when needed.

retrieve_framework_document

Retrieve an inference pipeline by ID.

retrieve_inference_pipeline

Retrieve a rule by ID.

retrieve_rule

Retrieve a single rule result by ID.

retrieve_rule_result

Retrieve one test in full, including its complete thresholds and rubric. ``list_tests`` deliberately summarizes thresholds to one line to save tokens, so this is the way to see what a test actually asserts -- the full threshold list, and for LLM-rubric tests the evaluation prompt itself. Read this before calling ``update_test``, so you preserve the fields you don't mean to change.

retrieve_test

Search the Openlayer documentation for relevant information. Use as a last resort when other tools don't provide what's needed. Returns a list of results, each with 'title', 'link', and 'content'.

search_openlayer_docs

Patch fields onto an existing production row -- e.g. backfill ground truth. The main use is labeling: tests that compare predictions against ground truth cannot evaluate rows where the ground-truth column is empty, so filling it in is what makes those rows measurable. This modifies live production data. Only the keys present in ``row`` are written; other columns are left alone. Confirm the target with ``get_data_source_row`` first.

update_data_source_row