Apastra — PromptOps for AI Teams

Ship AI prompts with the same discipline as code. Version them, test them, catch regressions — all locally, using the agent in your IDE.

Apastra is a file-based PromptOps framework that treats AI prompts as versioned software assets. Prompts, test cases, scoring rules, and quality baselines are all files in your repo — and your IDE agent is the harness that runs evaluations.

No cloud platform required. No CI needed to get started. Just files and your agent.

Quickstart

Install skills and run your first evaluation in 5 minutes

Core Concepts

Understand prompt specs, datasets, evaluators, suites, and baselines

Skills Reference

Explore all available apastra skills for your IDE agent

Writing Evals

Learn to write effective evaluations that catch real regressions

How it works

Your IDE agent (Claude, Cursor, Amp, Codex, and many more) reads the protocol files and executes the evaluation workflow — no external runtime, no API keys to configure.

1. Install skills

   Add apastra skills to your IDE agent with a single command:

   npx skills add BintzGavin/apastra --all --full-depth -y
   
2. Scaffold your first prompt

   Ask your agent to create a prompt spec, dataset, evaluator, and test suite:

   “Use the apastra-scaffold skill to create a prompt spec, dataset, evaluator, and suite for summarizing text”

3. Run your first eval

   Ask your agent to evaluate the prompt:

   “Use the apastra-eval skill to run the summarize-smoke suite”

   Your agent reads the suite, runs each test case through the model, scores results, and reports pass/fail.

4. Set a baseline

   Lock in your current quality level so future changes are automatically compared:

   “Use the apastra-baseline skill to set the current results as the baseline”

What you get

Prompt versioning

Prompt specs are YAML files with stable IDs, variable schemas, and output contracts — versioned in Git like any other code

Automated evals

Your IDE agent runs test suites, scores outputs, and reports pass/fail — no external platform needed

Regression detection

Compare new results against known-good baselines to catch quality drops before they ship

Schema validation

56 JSON schemas ensure all your promptops files are correctly formatted and machine-readable

Rich assertions

Deterministic, AI-graded, and performance assertion types — from contains to llm-rubric

CI integration

Optional GitHub Actions workflows for regression gating, promotion, and immutable releases

Apastra is local-first by design. You can use it entirely without CI or cloud infrastructure. When you’re ready, the apastra-setup-ci skill upgrades your workflow to GitHub Actions in minutes.