> ## Documentation Index
> Fetch the complete documentation index at: https://bintzgavin-apastra-14.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# CI integration

> Automate prompt evaluation with GitHub Actions — from a simple two-workflow setup to full enterprise governance.

Local evals with your IDE agent are the starting point. CI integration is the next step: it makes regression detection automatic, gates merges on passing checks, and creates an audit trail for every prompt change.

## When to upgrade from local to CI

| Signal                               | What it means                                                                               |
| ------------------------------------ | ------------------------------------------------------------------------------------------- |
| More than one engineer edits prompts | You need merge gates — local evals don't protect the shared branch                          |
| Prompt changes ship without review   | You need a required status check that blocks merge                                          |
| You want an audit trail              | CI creates a durable record of every eval run tied to a commit SHA                          |
| Baselines need to be shared          | CI writes baselines to the `promptops-artifacts` branch so everyone uses the same reference |

If you're a solo builder or in early prototyping, local evals are sufficient. Upgrade to CI when the team or stakes grow.

## Basic CI: two workflows

For most teams, two workflows cover the full loop — eval on pull requests, release on tag push.

<Tabs>
  <Tab title="prompt-eval.yml">
    **Trigger:** Pull requests with changes to `promptops/**`

    **What it does:** Runs the regression gate on every PR that touches prompt specs, datasets, evaluators, or policies. Blocks merge if a regression is detected.

    ```yaml theme={null}
    name: Prompt Eval
    on:
      pull_request:
        paths:
          - 'promptops/**'

    jobs:
      eval:
        uses: ./.github/workflows/regression-gate.yml
    ```

    This workflow delegates to `regression-gate.yml` (the reusable workflow), which:

    1. Detects changed files in `promptops/harnesses/**`, `promptops/prompts/**`, `promptops/datasets/**`, and `promptops/policies/*.yaml`
    2. Skips if no evaluable files changed
    3. Fetches the latest regression report from the `promptops-artifacts` branch
    4. Evaluates the report status and posts a step summary with per-metric evidence
    5. Checks cost budgets against suite `budgets.cost_budget` limits
    6. Exits non-zero (blocking merge) if the report status is not `pass`
  </Tab>

  <Tab title="prompt-release.yml">
    **Trigger:** Tag pushes (any tag matching `*`)

    **What it does:** Creates an immutable GitHub Release with a signed content digest for the `promptops/` bundle.

    ```yaml theme={null}
    name: Prompt Release
    on:
      push:
        tags:
          - '*'

    jobs:
      release:
        uses: ./.github/workflows/immutable-release.yml
    ```

    This workflow delegates to `immutable-release.yml`, which:

    1. Packages `promptops/` into `promptops.tar.gz`
    2. Computes a SHA-256 digest of the bundle
    3. Attests build provenance using `actions/attest-build-provenance`
    4. Creates a GitHub Release with the bundle as an asset and the digest in the release notes
  </Tab>
</Tabs>

## Full CI: six workflows

For enterprise teams needing fine-grained control, apastra ships six individual workflows. Each has a distinct responsibility:

| Workflow                | Trigger                                                    | What it does                                                                                                                                                                 |
| ----------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `regression-gate.yml`   | Pull requests to `main`; also callable via `workflow_call` | Detects changed files, fetches the regression report from artifacts branch, evaluates pass/fail, checks cost budgets, posts step summary, blocks merge on failure            |
| `promote.yml`           | Manual dispatch (`workflow_dispatch`) or release publish   | Verifies an approval state record exists for the digest, generates an append-only promotion record, commits it to the `promptops-artifacts` branch, then calls `deliver.yml` |
| `deliver.yml`           | Called by `promote.yml` via `workflow_call`                | Reads the promotion record, resolves the channel and digest, iterates over `promptops/delivery/*.yaml` targets, and executes the sync for each matching target               |
| `immutable-release.yml` | Tag push (`*`); also callable via `workflow_call`          | Packages `promptops/`, computes digest, attests build provenance, creates an immutable GitHub Release with the bundle and digest                                             |
| `auto-merge.yml`        | Pull request opened, reopened, or synchronized             | Resolves `PROGRESS.md` merge conflicts automatically, then enables squash auto-merge for PRs that pass all required checks                                                   |
| `prompt-eval.yml`       | Pull requests touching `promptops/**`                      | Entry point for the basic CI setup — delegates to `regression-gate.yml`                                                                                                      |

<Note>
  `regression-gate.yml`, `immutable-release.yml`, and `deliver.yml` are reusable workflows (they declare `workflow_call` as a trigger). This means you can call them from other workflows in the same repo or from workflows in other repos, which lets a platform team standardize PromptOps across many repositories.
</Note>

## What the regression gate does in detail

The regression gate is the most important workflow for day-to-day governance. Here is what happens step by step when a PR is opened:

<Steps>
  <Step title="Detect changed files">
    The workflow uses `tj-actions/changed-files` to detect whether any of the following changed:

    * `promptops/harnesses/**`
    * `promptops/prompts/**`
    * `promptops/datasets/**`
    * `promptops/policies/*.yaml`

    If none of these changed, the gate is skipped with a log message. Non-prompt changes (docs, code, config) are not gated.
  </Step>

  <Step title="Fetch the artifacts branch">
    When evaluable files changed, the workflow fetches `reports/regression_report.json` and `reports/run_manifest.json` from the `promptops-artifacts` branch. These were written by the most recent eval run.
  </Step>

  <Step title="Evaluate the regression report">
    The workflow reads the report and posts a step summary table with columns: Metric, Status, Candidate, Baseline, Delta, Message. Any failing metrics are annotated as errors in the PR.
  </Step>

  <Step title="Check cost budgets">
    If a `run_manifest.json` includes a `total_cost` field, the workflow compares it against the `budgets.cost_budget` declared in each suite file. A cost overrun fails the gate.
  </Step>

  <Step title="Block or pass">
    If `report.status == "pass"`, the workflow exits 0 (green check). Otherwise it exits 1 (red check), which fails the required status check and blocks merge on protected branches.
  </Step>
</Steps>

## CODEOWNERS for prompt governance

Use a `CODEOWNERS` file to require human review of any prompt, policy, or evaluator change:

```
# Prompt specs require review from the AI quality team
promptops/prompts/         @your-org/ai-quality
promptops/policies/        @your-org/ai-quality
promptops/evaluators/      @your-org/ai-quality

# Delivery targets require platform team review
promptops/delivery/        @your-org/platform
```

With CODEOWNERS in place:

* Any PR that modifies a file in `promptops/prompts/` requires at least one approved review from `@your-org/ai-quality` before merge.
* This is enforced by GitHub branch protection rules — not just convention.

## Branch protection: required status checks

Configure branch protection on `main` to require the regression gate before merge:

1. Go to **Settings → Branches → Branch protection rules**.
2. Add a rule for `main`.
3. Enable **Require status checks to pass before merging**.
4. Add `gate` (the job name from `regression-gate.yml`) as a required check.
5. Optionally enable **Require a pull request before merging** and **Require review from Code Owners**.

With this configuration, no PR can merge to `main` unless:

* The regression gate job passed (no regression detected, no cost budget exceeded)
* At least one CODEOWNERS reviewer approved (if CODEOWNERS is configured)

## Auto-merge for passing PRs

The `auto-merge.yml` workflow enables squash auto-merge for PRs that pass all required checks. It fires on `pull_request` events (opened, reopened, synchronized) and:

1. Resolves any `PROGRESS.md` merge conflicts automatically using a union merge strategy.
2. Calls `gh pr merge --auto --squash` to enable auto-merge once all required checks are green.

<Info>
  In the apastra repo, auto-merge is scoped to PRs from `google-labs-jules` and `BintzGavin`. For your own repo, update the `if:` conditions to match your autonomous agent's login or a specific label.
</Info>

## Artifacts branch: keeping derived data out of main

Regression reports, run manifests, promotion records, and baselines are derived data — they should not live on `main` alongside your source files. Apastra uses a separate `promptops-artifacts` branch as an append-only store:

```
# branch: promptops-artifacts
reports/
  regression_report.json
  run_manifest.json
approvals/
  <approval-record>.json
promotions/
  <timestamp>-<id>.json
```

The regression gate fetches from this branch at runtime. The promote workflow commits to this branch. This keeps the main branch clean and prevents merge conflicts on derived files.