> ## Documentation Index
> Fetch the complete documentation index at: https://bintzgavin-apastra-14.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Schema reference

> Reference for all 56 JSON schemas that validate apastra protocol files, organized by category with field-level documentation for core schemas.

Apastra ships 56 JSON schemas that validate every file type in the protocol. Schemas ensure your prompts, datasets, evaluators, and run artifacts are machine-readable by any agent or harness — not just your own.

All schemas use JSON Schema draft 2020-12.

<Note>
  Your agent validates files against these schemas when you run the `apastra-validate` skill. The `schema-validation.yml` CI workflow also validates changed files on every pull request.
</Note>

***

## Schema categories

<Tabs>
  <Tab title="Core protocol">
    | Schema file                    | What it validates                                               |
    | ------------------------------ | --------------------------------------------------------------- |
    | `prompt-spec.schema.json`      | Prompt template + variable schema + output contract             |
    | `dataset-manifest.schema.json` | Dataset identity, version, digest, provenance                   |
    | `dataset-case.schema.json`     | A single JSONL test case with inputs and inline assertions      |
    | `evaluator.schema.json`        | Scoring rules (deterministic, schema, judge, human)             |
    | `suite.schema.json`            | Benchmark suite: datasets, evaluators, model matrix, thresholds |
    | `quick-eval.schema.json`       | Single-file eval combining prompt, cases, and assertions        |
  </Tab>

  <Tab title="Execution">
    | Schema file                 | What it validates                                          |
    | --------------------------- | ---------------------------------------------------------- |
    | `run-request.schema.json`   | Immutable work order for triggering a run                  |
    | `run-manifest.schema.json`  | Resolved digests, harness, model IDs, env metadata, status |
    | `scorecard.schema.json`     | Normalized metrics, metric definitions, variance           |
    | `run-case.schema.json`      | Per-case result record                                     |
    | `run-artifact.schema.json`  | Complete run artifact directory structure                  |
    | `run-failures.schema.json`  | Structured failure records from a run                      |
    | `artifact-refs.schema.json` | URIs + digests pointing to large raw artifacts             |
  </Tab>

  <Tab title="Quality">
    | Schema file                           | What it validates                                          |
    | ------------------------------------- | ---------------------------------------------------------- |
    | `baseline.schema.json`                | Named reference to a known-good scorecard                  |
    | `regression-policy.schema.json`       | Per-metric rules: floors, deltas, directionality, severity |
    | `regression-report.schema.json`       | Candidate vs baseline comparison outcome + evidence        |
    | `comparison-scorecard.schema.json`    | Side-by-side scorecard comparison                          |
    | `drift-report.schema.json`            | Canary suite drift detection report                        |
    | `flake-quarantine-record.schema.json` | Record of a quarantined flaky test case                    |
  </Tab>

  <Tab title="Governance">
    | Schema file                           | What it validates                               |
    | ------------------------------------- | ----------------------------------------------- |
    | `promotion-record.schema.json`        | Append-only binding: digest → channel           |
    | `approval-state.schema.json`          | Human review decision record                    |
    | `delivery-target.schema.json`         | Downstream sync target configuration            |
    | `delivery-target-receipt.schema.json` | Confirmation that a sync completed              |
    | `consumption-manifest.schema.json`    | App-side prompt pins, overrides, model defaults |
    | `policy-exception-record.schema.json` | Documented exception to a governance policy     |
    | `audit-report.schema.json`            | Audit trail record                              |
  </Tab>

  <Tab title="Packaging">
    | Schema file                                | What it validates                                            |
    | ------------------------------------------ | ------------------------------------------------------------ |
    | `prompt-package.schema.json`               | Immutable bundle of prompt specs with manifest and digest    |
    | `provider-artifact.schema.json`            | Distribution wrapper (git ref, release asset, OCI, npm/PyPI) |
    | `harness-adapter.schema.json`              | How to invoke a harness + capabilities + env vars            |
    | `release-descriptor.schema.json`           | Signed release descriptor for delivery to internal APIs      |
    | `provenance-attestation.schema.json`       | SLSA-style build provenance attestation                      |
    | `trusted-publisher-provenance.schema.json` | Trusted publisher badge provenance data                      |
  </Tab>

  <Tab title="Registry (future)">
    | Schema file                                          | What it validates                            |
    | ---------------------------------------------------- | -------------------------------------------- |
    | `submission-record.schema.json`                      | Package submission to the public registry    |
    | `moderation-decision-record.schema.json`             | Moderation outcome for a submitted package   |
    | `moderation-approval-for-public-listing.schema.json` | Approval to list a package publicly          |
    | `moderation-escalation-record.schema.json`           | Escalation record for human moderator review |
    | `reject-record.schema.json`                          | Package rejection record with reasons        |
    | `community-report-record.schema.json`                | Community abuse/moderation report            |
    | `namespace-claim-record.schema.json`                 | Namespace ownership claim                    |
    | `ownership-dispute-record.schema.json`               | Dispute over package namespace ownership     |
    | `deprecation-record.schema.json`                     | Package deprecation notice                   |
    | `takedown-record.schema.json`                        | Package takedown record                      |
    | `takedown-appeal-record.schema.json`                 | Appeal of a takedown decision                |
    | `emergency-takedown-decision.schema.json`            | Emergency takedown decision record           |
    | `vulnerability-flag-record.schema.json`              | Security vulnerability flag on a package     |
    | `mirror-sync-receipt.schema.json`                    | Confirmation of a mirror registry sync       |
    | `community-prompt-pack.schema.json`                  | Curated community prompt pack definition     |
    | `agent-skill.schema.json`                            | Agent skill definition (SKILL.md format)     |
  </Tab>
</Tabs>

***

## Core schema field reference

### `prompt-spec.schema.json`

Source-of-truth prompt definition. Required in every apastra project.

<ParamField path="id" type="string" required>
  Stable identifier for the prompt. Use a namespaced slug such as `my-app/summarize-v1`. Renaming breaks consumption manifest pins.
</ParamField>

<ParamField path="variables" type="object" required>
  Map of variable names to JSON Schema type objects. Each key is a template placeholder; each value defines the type.

  ```yaml theme={null}
  variables:
    text: { type: string }
    max_length: { type: string }
  ```
</ParamField>

<ParamField path="template" type="string | object | array" required>
  The prompt template. For completion models, use a string with `{{variable}}` placeholders. For chat models, use an array of message objects.

  ```yaml theme={null}
  template: "Summarize: {{text}}"
  ```
</ParamField>

<ParamField path="output_contract" type="object">
  JSON Schema defining the expected output structure. Used by `schema` evaluators to validate model responses.
</ParamField>

<ParamField path="tool_contract" type="object">
  JSON Schema defining expected tool calling structure and available tools. Required if the prompt uses function calling.
</ParamField>

<ParamField path="metadata" type="object">
  Arbitrary key-value pairs such as `author`, `intent`, and `tags`.
</ParamField>

***

### `dataset-manifest.schema.json`

Declares a dataset's identity and content digest for reproducibility.

<ParamField path="id" type="string" required>
  Stable identifier for the dataset.
</ParamField>

<ParamField path="version" type="string" required>
  Semantic version or revision of the dataset. Treat dataset edits as new versions.
</ParamField>

<ParamField path="digest" type="string" required>
  SHA-256 content digest of the `.jsonl` file. Format: `sha256:<hex>`. See [digest convention](#digest-convention).
</ParamField>

<ParamField path="schema_version" type="string" required>
  Version of the `dataset-case` schema used by the JSONL file.
</ParamField>

<ParamField path="provenance" type="object">
  Information about the origin of the dataset.
</ParamField>

***

### `dataset-case.schema.json`

A single test case — one line in a JSONL dataset file.

<ParamField path="case_id" type="string" required>
  Stable identifier for the test case. Never change existing `case_id` values; add new cases instead.
</ParamField>

<ParamField path="inputs" type="object" required>
  Map of variable names to input values, matching the prompt spec's `variables` schema.
</ParamField>

<ParamField path="assert" type="array">
  Array of inline assertion objects. Each object has `type` (string) and `value` (any). See [assertion types](/reference/assertion-types) for the full list.
</ParamField>

<ParamField path="expected_outputs" type="object">
  Expected output values for evaluator scoring (e.g., `should_contain` keyword lists).
</ParamField>

<ParamField path="metadata" type="object">
  Arbitrary metadata for the case (e.g., tags, difficulty, domain).
</ParamField>

***

### `evaluator.schema.json`

Scoring definition for a suite.

<ParamField path="id" type="string" required>
  Stable identifier for the evaluator.
</ParamField>

<ParamField path="type" type="string" required>
  Evaluator type. One of: `deterministic`, `schema`, `judge`, `human`.
</ParamField>

<ParamField path="metrics" type="array" required>
  Array of metric names produced by this evaluator (e.g., `["keyword_recall"]`). At least one is required.
</ParamField>

<ParamField path="config" type="object">
  Evaluator-type-specific configuration. For `judge` evaluators, this includes `rubric` and model details. For `schema` evaluators, this includes the target JSON Schema.
</ParamField>

<ParamField path="metric_versions" type="object">
  Mapping of metric names to their semantic versions. Increment when changing how a metric is computed to preserve historical comparability.
</ParamField>

<ParamField path="digest" type="string">
  SHA-256 hash of the evaluator content. Format: `sha256:<64 hex chars>`.
</ParamField>

***

### `suite.schema.json`

Benchmark suite configuration.

<ParamField path="id" type="string" required>
  Stable identifier for the suite.
</ParamField>

<ParamField path="name" type="string" required>
  Human-readable name.
</ParamField>

<ParamField path="datasets" type="array" required>
  Array of dataset IDs (at least one). Each ID maps to a `.jsonl` file in `promptops/datasets/`.
</ParamField>

<ParamField path="evaluators" type="array" required>
  Array of evaluator IDs (at least one). Each ID maps to a `.yaml` file in `promptops/evaluators/`.
</ParamField>

<ParamField path="model_matrix" type="array" required>
  Array of model or provider identifiers to run the suite against. Use `"default"` to mean the agent's own model.
</ParamField>

<ParamField path="tier" type="string">
  Execution tier. One of: `smoke`, `regression`, `full`, `release-candidate`. Default: `smoke`.
</ParamField>

<ParamField path="trials" type="integer">
  Number of times to run each case for variance measurement. Default: `1`. Use `3+` for regression suites.
</ParamField>

<ParamField path="budgets" type="object">
  Cost and time limits. Supports `cost_budget` (dollars) and `time` (seconds).
</ParamField>

<ParamField path="thresholds" type="object">
  Pass/fail criteria. Keys are metric names; values are minimum acceptable scores.

  ```yaml theme={null}
  thresholds:
    keyword_recall: 0.6
    pass_rate: 1.0
  ```
</ParamField>

***

### `quick-eval.schema.json`

Single-file evaluation format combining prompt, cases, and assertions.

<ParamField path="id" type="string" required>
  Stable identifier for the quick eval.
</ParamField>

<ParamField path="prompt" type="string" required>
  The prompt template with `{{variable}}` placeholders.
</ParamField>

<ParamField path="cases" type="array" required>
  Array of test cases. Each case follows the `dataset-case` schema with `id`, `inputs`, and `assert`.
</ParamField>

<ParamField path="thresholds" type="object">
  Pass/fail thresholds, typically `pass_rate: 1.0`.
</ParamField>

***

### `run-manifest.schema.json`

Durable metadata record for a completed run. Written by the harness.

<ParamField path="input_refs" type="object" required>
  References to input files (suite, prompt, dataset, evaluator IDs).
</ParamField>

<ParamField path="resolved_digests" type="object" required>
  Content digests of all resolved inputs at run time, enabling replay.
</ParamField>

<ParamField path="timestamps" type="object" required>
  Run start and end times.
</ParamField>

<ParamField path="harness_identifier" type="string" required>
  Identifier of the execution environment. Common values: `claude-code`, `antigravity`, `cursor`, `copilot`, `api`, `github-actions`, `jules`.
</ParamField>

<ParamField path="harness_version" type="string" required>
  Version of the harness. The same model in different harnesses can produce different outputs.
</ParamField>

<ParamField path="model_ids" type="array" required>
  Array of model identifiers used in the run.
</ParamField>

<ParamField path="sampling_config" type="object" required>
  Temperature, top-p, and other sampling parameters used.
</ParamField>

<ParamField path="environment" type="object" required>
  Environment metadata for reproduction attempts.
</ParamField>

<ParamField path="status" type="string" required>
  Run outcome. Typically `pass` or `fail`.
</ParamField>

<ParamField path="total_cost" type="number">
  Total cost of the run in dollars (input tokens × price + output tokens × price).
</ParamField>

<ParamField path="provenance" type="object">
  SLSA-style provenance metadata: `builder.id`, `buildType`, `invocation`, and `metadata`.
</ParamField>

***

### `scorecard.schema.json`

Normalized metrics summary for a run.

<ParamField path="normalized_metrics" type="object" required>
  Mapping of metric names to their aggregated values (0–1 scale for most metrics).

  ```json theme={null}
  {"keyword_recall": 0.85, "pass_rate": 1.0}
  ```
</ParamField>

<ParamField path="metric_definitions" type="object" required>
  Metadata for each metric. Each entry requires `version` and optionally includes `description` and `direction`.

  ```json theme={null}
  {
    "keyword_recall": {
      "version": "1.0",
      "description": "Fraction of expected keywords found in output",
      "direction": "higher_is_better"
    }
  }
  ```
</ParamField>

<ParamField path="variance" type="object">
  Variance data if `trials > 1` was configured in the suite.
</ParamField>

<ParamField path="flake_rates" type="object">
  Mapping of metric names to their observed flake rates.
</ParamField>

***

### `baseline.schema.json`

Reference to a known-good scorecard for regression comparison.

<ParamField path="baseline_id" type="string" required>
  Identifier for this baseline record.
</ParamField>

<ParamField path="run_digest" type="string" required>
  Content digest of the reference run's scorecard.
</ParamField>

<ParamField path="created_at" type="string" required>
  ISO 8601 timestamp when the baseline was established.
</ParamField>

<ParamField path="description" type="string">
  Human-readable description (e.g., "post-v2-launch baseline").
</ParamField>

***

### `regression-policy.schema.json`

Defines how candidate scorecards are compared against baselines.

<ParamField path="baseline" type="string" required>
  Baseline reference rule (e.g., `"prod current"`, `"last-rc-passing-run"`).
</ParamField>

<ParamField path="rules" type="array" required>
  Array of per-metric rule objects. Each rule requires `metric` and `severity`.

  ```yaml theme={null}
  rules:
    - metric: keyword_recall
      floor: 0.5
      allowed_delta: 0.1
      direction: higher_is_better
      severity: blocker
  ```
</ParamField>

Rule fields:

<ParamField path="rules[].metric" type="string" required>
  Metric name to evaluate.
</ParamField>

<ParamField path="rules[].severity" type="string" required>
  `blocker` — fails the check and blocks merge. `warning` — reported but does not block.
</ParamField>

<ParamField path="rules[].floor" type="number">
  Absolute minimum acceptable value for this metric.
</ParamField>

<ParamField path="rules[].allowed_delta" type="number">
  Maximum allowed drop from the baseline value.
</ParamField>

<ParamField path="rules[].direction" type="string">
  `higher_is_better` or `lower_is_better`. Controls which direction a delta is treated as a regression.
</ParamField>

***

### `consumption-manifest.schema.json`

App-side file declaring prompt pins and resolution overrides.

<ParamField path="version" type="string" required>
  Version of the consumption manifest format.
</ParamField>

<ParamField path="prompts" type="object" required>
  Mapping of local prompt names to resolution configurations. Each entry requires `id` and optionally includes `pin`, `override`, and `model`.

  ```yaml theme={null}
  prompts:
    summarize-v1:
      id: summarize-v1
      pin: "abc123"
  ```
</ParamField>

<ParamField path="prompts[].id" type="string" required>
  Stable prompt ID to resolve.
</ParamField>

<ParamField path="prompts[].pin" type="string">
  Git ref, commit SHA, semver range, or packaged artifact reference. See [resolver](/reference/resolver) for supported pin formats.
</ParamField>

<ParamField path="prompts[].override" type="string">
  Local file path overriding resolution. Used for local-linked development.
</ParamField>

<ParamField path="prompts[].model" type="string">
  Override the default model for this specific prompt.
</ParamField>

<ParamField path="defaults" type="object">
  Global fallbacks: `model` and `provider`.
</ParamField>

***

### `promotion-record.schema.json`

Append-only record binding an approved version to a delivery channel.

<ParamField path="version" type="string" required>
  The approved version being promoted.
</ParamField>

<ParamField path="channel" type="string" required>
  Target channel (e.g., `prod`, `staging`, `release`).
</ParamField>

<ParamField path="digest" type="string" required>
  Content digest of the promoted version.
</ParamField>

<ParamField path="evidence" type="object">
  Links to supporting evidence (e.g., `run_id` of the release-candidate run).
</ParamField>

<ParamField path="timestamp" type="string">
  ISO 8601 timestamp of the promotion event.
</ParamField>

***

### `delivery-target.schema.json`

Declarative configuration for a downstream sync target.

<ParamField path="type" type="string" required>
  Target type (e.g., `github_pr`, `oci_registry`).
</ParamField>

<ParamField path="repo" type="string" required>
  Target repository (for `github_pr` type).
</ParamField>

***

## Digest convention

All `digest` fields in apastra schemas use SHA-256 computed over canonicalized content.

### Canonicalization rules

<AccordionGroup>
  <Accordion title="JSON files (.json)">
    1. Parse the JSON.
    2. Sort all keys alphabetically (recursively).
    3. Remove all insignificant whitespace.
    4. This is equivalent to: `jq -cSM . <file>`
  </Accordion>

  <Accordion title="YAML files (.yaml, .yml)">
    1. Parse the YAML into a JSON object.
    2. Apply the same canonicalization as JSON files.
  </Accordion>

  <Accordion title="JSONL files (.jsonl)">
    1. Parse each line as JSON.
    2. Canonicalize each line independently.
    3. Rejoin lines with a single `\n` between each.
    4. Hash the resulting string.
  </Accordion>
</AccordionGroup>

### Digest format

```
sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
```

All `digest` fields that use the pattern validator require exactly `sha256:` followed by 64 lowercase hex characters.

***

## Using schemas for validation

### With the validate skill

Ask your agent:

```
Use the apastra-validate skill to validate all promptops files
```

### With the CLI

```bash theme={null}
npm install -g ajv-formats ajv-cli
ajv validate -s promptops/schemas/prompt-spec.schema.json -d promptops/prompts/summarize-v1.yaml
```

### In CI

The `schema-validation.yml` workflow validates changed prompt and dataset files on every pull request automatically. See [GitHub workflows reference](/reference/github-workflows#schema-validation) for details.