> ## Documentation Index
> Fetch the complete documentation index at: https://bintzgavin-apastra-14.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# apastra-baseline

> Establish and manage evaluation baselines for regression detection. A baseline is a saved scorecard from a passing run — future evals compare against it to catch quality drops.

## Installation

```bash theme={null}
npx skills add BintzGavin/apastra/skills/baseline
```

## What is a baseline?

A baseline is a snapshot of a scorecard from a known-good evaluation run. Once established, every future eval for that suite is compared against it. If a prompt change causes quality to drop beyond the allowed thresholds defined in your regression policy, the eval reports a regression.

<Info>
  Baselines are stored as JSON files in `derived-index/baselines/`. They are never deleted — when you update a baseline, the previous one is archived with a timestamp suffix.
</Info>

## How to invoke

Ask your agent:

> "Use the apastra-baseline skill to set the current results as the baseline for \[suite-name]"

## Establishing a baseline

<Steps>
  <Step title="Locate the scorecard">
    Your agent finds the most recent run for the target suite in `promptops/runs/`. It looks for the latest directory matching `<suite-id>-*` and reads its `scorecard.json`.

    If no recent run exists, your agent will prompt you to run the **apastra-eval** skill first.
  </Step>

  <Step title="Create the baseline file">
    Your agent writes the baseline to `derived-index/baselines/<suite-id>.json`:

    ```json theme={null}
    {
      "suite_id": "summarize-smoke",
      "established_at": "2026-03-11T12:00:00Z",
      "source_run": "summarize-smoke-2026-03-11-120000",
      "scorecard": {
        "normalized_metrics": {
          "keyword_recall": 0.85
        },
        "metric_definitions": {
          "keyword_recall": {
            "description": "Fraction of expected keywords found in output",
            "version": "1.0",
            "direction": "higher_is_better"
          }
        }
      }
    }
    ```
  </Step>

  <Step title="Confirm">
    Your agent reports what was established:

    ```text theme={null}
    Baseline established ✅

    Suite: summarize-smoke
    Source run: summarize-smoke-2026-03-11-120000
    Metrics:
      keyword_recall: 0.85

    Saved to: derived-index/baselines/summarize-smoke.json
    ```
  </Step>
</Steps>

## Updating a baseline

When you've verified that a prompt improvement is intentional and you want to raise the bar, you can update the baseline.

Your agent follows an append-friendly model:

1. Renames the existing baseline to `<suite-id>-<timestamp>.json` (for example, `summarize-smoke-2026-03-10.json`) as an archive
2. Writes the new baseline to `derived-index/baselines/<suite-id>.json`
3. Reports both the old and new metric values so the change is visible

```text theme={null}
Baseline updated ✅

Suite: summarize-smoke
Previous baseline: keyword_recall=0.80 (archived to summarize-smoke-2026-03-10.json)
New baseline: keyword_recall=0.85

The eval skill will now compare future runs against the new baseline.
```

## Rolling back a baseline

If a regression surfaces and you need to undo a baseline update, ask your agent to restore a prior baseline:

> "Use the apastra-baseline skill to roll back the summarize-smoke baseline"

Your agent copies the archived baseline file (for example, `summarize-smoke-2026-03-10.json`) back to `derived-index/baselines/summarize-smoke.json`. This promotes the prior scorecard as the active baseline without deleting any records.

## Baseline file location

Active baselines are always at:

```
derived-index/baselines/<suite-id>.json
```

Archived (superseded) baselines are stored alongside them with a timestamp suffix:

```
derived-index/baselines/<suite-id>-<YYYY-MM-DD>.json
```

## Relationship to regression policies

The baseline file contains the reference metrics. The regression policy (`promptops/policies/regression.yaml`) defines how much deviation is allowed before an eval is marked as a regression:

```yaml theme={null}
baseline: "prod-current"
rules:
  - metric: keyword_recall
    floor: 0.5
    allowed_delta: 0.1
    direction: higher_is_better
    severity: blocker
```

During an eval, your agent reads both files and compares:

* For `higher_is_better` metrics: regression if candidate \< (baseline − allowed\_delta) or candidate \< floor
* For `lower_is_better` metrics: regression if candidate > (baseline + allowed\_delta) or candidate > floor

Rules with `severity: blocker` fail the eval. Rules with `severity: warning` are reported but do not block.

## Rules

<Warning>
  Never establish a baseline from a failing run. A baseline represents a quality floor — if you baseline a failing scorecard, all future comparisons will be measured against a poor result.
</Warning>

* **Never delete a baseline** — archive it with a timestamp suffix
* **Only baseline passing runs** — the scorecard must have passed all suite thresholds
* **One active baseline per suite** — the active baseline is always `<suite-id>.json`
* **Baselines are immutable once set** — updating means archiving the old file and writing a new one