InferGauge

Answers "did my latest release regress?", "did it get worse?" or "how does this run compare to the last one?". Compares one run with a baseline run metric by metric - latency, TTFT, inter-token latency, error rate, throughput, goodput, quality, cost per request - and flags each metric that worsened by more than max_regression_pct. That threshold defaults to 10% when the user names none; the result reports which threshold was applied and whether it was the default, so say so when giving the verdict. Omit run_id for the most recent run, or pass test_name; omit baseline_run_id to compare against the nearest earlier completed run of the same test. Omitting both answers the regression question in one call.

Generates a CI performance gate: pipeline configuration that fails a pull request when an InferGauge test shows performance got worse. Use it when the user asks for a CI check, performance gate or regression check in their pipeline. How the gate works: each push to `branch` runs the test on the customer's own CI runner and saves the result as the baseline. Each pull request into `branch` restores that baseline, runs the test again, and fails if any metric (latency, TTFT, inter-token latency, error rate, throughput, goodput, quality, cost per request) is more than max_regression_pct worse, or if the test breaches one of its own SLAs. This tool only returns text. It commits nothing, runs no test, creates no credentials and never contacts the user's CI system or repository. The workflow needs one secret, INFERGAUGE_TOKEN: a revocable token the user creates in the console under Settings, API Tokens. Show the workflow to the user unchanged, since every line is needed for it to run.

Build a NEW load-test config when the user asks to create, set up or write a test. Not for questions about tests already run - use get_test_results or compare_runs for those. Validates the request against the account's plan and, if allowed, returns runnable YAML plus the exact command to run it on the customer's own machine. Starts nothing and uses none of the monthly run quota. Refuses outright, with no YAML, if the request exceeds the plan: too many users, too long a duration, a test type or endpoint kind the plan doesn't include, or the monthly run quota already used up. Leave provider_kind as "simulator" unless the user gives their endpoint details unprompted: the YAML then carries commented placeholders for base URL, model, key variable and prompt, which the customer fills in locally. api_key_env is an environment-variable NAME the customer's machine reads at run time - never an actual key, password or other credential value; anything that isn't a plausible variable name is rejected.

Current account's tier, entitlement limits, and usage this billing period.

Results of a test the customer has already run: score, SLA pass or fail, latency, TTFT, inter-token latency, throughput, error rate, goodput, quality and cost, plus plain-language insights. The insights answer analysis questions directly - including where the application saturates under load (the concurrent-user level where latency inflects, and whether that point lies inside or beyond the tested range), rate limiting, and cost overruns. Use this, not create_test, for any question about how a test went. Omit run_id for the most recent run, or pass test_name to pick the latest run of a named test. Raw provider error text is redacted. For a test still in progress, use get_test_status.

Status and, if still running, a live progress snapshot for one of the account's own runs. Omit run_id for the most recent run (optionally narrowed with test_name, for an account running more than one named test).

Metadata (no results) for the account's most recent runs, newest first - use get_test_results for any one run's full numbers.