Integrate orchagent into your CI/CD pipeline to automatically test and deploy agents on every push.
GitHub Actions Workflow
Create .github/workflows/deploy-agent.yml:
name: Deploy Agent
on:
push:
branches: [main]
paths:
- 'my-agent/**'
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install orchagent CLI
run: npm install -g @orchagent/cli
- name: Run agent tests
working-directory: my-agent
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: orch test
deploy:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install orchagent CLI
run: npm install -g @orchagent/cli
- name: Publish agent
working-directory: my-agent
env:
ORCHAGENT_API_KEY: ${{ secrets.ORCHAGENT_API_KEY }}
run: |
orch publish --dry-run
orch publish
Testing Orchestrators in CI
Orchestrator agents that call sub-agents can use mocked fixtures to test the full LLM reasoning loop without requiring live sub-agents. Create fixtures with a mocks field that maps custom tool names to deterministic responses:
my-orchestrator/
orchagent.json
prompt.md
schema.json
tests/
fixture-mock-happy-path.json
fixture-mock-error-handling.json
// tests/fixture-mock-happy-path.json
{
"description": "Orchestrator combines scan results correctly",
"input": {"code": "import requests"},
"mocks": {
"scan_secrets": {"findings": []},
"scan_deps": {"vulnerabilities": [{"name": "requests", "severity": "low"}]}
},
"expected_contains": ["requests", "low"]
}
orch test automatically detects mocked fixtures and runs the full agent loop with mock responses injected for custom tool calls. No additional CI configuration needed.
Mocked orchestration tests verify that the LLM correctly selects tools, processes their responses, and produces the expected output — all without network calls to deployed sub-agents. This makes them fast and reliable in CI.
Required Secrets
Add these to your GitHub repo under Settings > Secrets and variables > Actions:
| Secret | Description |
|---|
ORCHAGENT_API_KEY | Your orchagent API key (from orch login or dashboard) |
ANTHROPIC_API_KEY | LLM key for running fixture tests and mocked orchestration tests |
Deploying Multiple Agents
For monorepos with multiple agents, use --all to publish everything in dependency order:
jobs:
deploy:
steps:
- uses: actions/checkout@v4
- run: npm install -g @orchagent/cli
- name: Publish all agents (dependency order)
env:
ORCHAGENT_API_KEY: ${{ secrets.ORCHAGENT_API_KEY }}
run: orch publish --all
orchagent.json/SKILL.md files, builds a dependency graph, and publishes leaf-first. If any agent fails, the batch stops.
Alternative: For more control, use a matrix strategy:
jobs:
deploy:
strategy:
matrix:
agent: [leak-finder, vuln-scanner, security-review]
steps:
- uses: actions/checkout@v4
- run: npm install -g @orchagent/cli
- run: orch publish
working-directory: agents/${{ matrix.agent }}
env:
ORCHAGENT_API_KEY: ${{ secrets.ORCHAGENT_API_KEY }}
Matrix strategies run agents in parallel — ensure leaf agents are published before orchestrators by setting max-parallel: 1 or splitting into separate jobs with dependencies.
Version Pinning
By default, publishing auto-increments the version (v1, v2, v3…). Schedules and webhook triggers auto-update to the latest version unless pinned.
To pin a schedule to a specific version:
orch schedule update <schedule-id> --pin-version
Dry Run in PRs
Add a dry-run check to pull requests:
on:
pull_request:
paths: ['my-agent/**']
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm install -g @orchagent/cli
- name: Validate agent
working-directory: my-agent
env:
ORCHAGENT_API_KEY: ${{ secrets.ORCHAGENT_API_KEY }}
run: orch publish --dry-run
Next Steps
