---
title: CI Pipeline Agent Testing
description: "Run AI agents inside Caged sandboxes as part of your GitHub Actions CI/CD pipeline."
---

# CI Pipeline Agent Testing

Use Caged sandboxes in CI to run AI agents that fix failing tests, generate code, or validate changes — all isolated and cost-controlled.

## Prerequisites

- A `CAGED_API_KEY` stored as a GitHub Actions secret
- Your agent's API key (e.g., `ANTHROPIC_API_KEY`) also in GitHub secrets

## GitHub Actions Workflow

```yaml
# .github/workflows/agent-fix.yml
name: AI Agent Fix

on:
  workflow_dispatch:
    inputs:
      task:
        description: "What should the agent do?"
        required: true
        default: "Fix all failing tests"

jobs:
  agent-sandbox:
    runs-on: ubuntu-latest
    steps:
      - name: Install Caged CLI
        run: curl -fsSL https://get.caged.dev | sh

      - name: Login to Caged
        run: |
          mkdir -p ~/.config/caged
          echo '{"api_url":"https://api.caged.dev","api_key":"${{ secrets.CAGED_API_KEY }}"}' > ~/.config/caged/config.json

      - name: Create sandbox and run agent
        id: sandbox
        run: |
          # Create sandbox with the repo
          SANDBOX_ID=$(caged run \
            --template node-20 \
            --cpus 2 \
            --memory 2048 \
            --repo ${{ github.server_url }}/${{ github.repository }} \
            --budget 5 \
            --env "ANTHROPIC_API_KEY=${{ secrets.ANTHROPIC_API_KEY }}" \
            --json | jq -r '.id')

          echo "sandbox_id=$SANDBOX_ID" >> "$GITHUB_OUTPUT"
          echo "Created sandbox: $SANDBOX_ID"

      - name: Run agent task
        run: |
          caged exec ${{ steps.sandbox.outputs.sandbox_id }} \
            "claude '${{ github.event.inputs.task }}'"

      - name: Extract results
        run: |
          # Get the git diff from the sandbox
          caged exec ${{ steps.sandbox.outputs.sandbox_id }} \
            "git diff" > agent-changes.patch

          # Check if there are changes
          if [ -s agent-changes.patch ]; then
            echo "Agent made changes — patch file saved"
          else
            echo "No changes made"
          fi

      - name: Cleanup sandbox
        if: always()
        run: caged destroy ${{ steps.sandbox.outputs.sandbox_id }} --force
```

## Auto-Fix on Test Failure

Run an agent automatically when tests fail:

```yaml
# .github/workflows/auto-fix.yml
name: Auto-Fix Tests

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test
        id: tests
        continue-on-error: true

      - name: Agent Fix (on failure)
        if: steps.tests.outcome == 'failure'
        run: |
          curl -fsSL https://get.caged.dev | sh
          mkdir -p ~/.config/caged
          echo '{"api_url":"https://api.caged.dev","api_key":"${{ secrets.CAGED_API_KEY }}"}' > ~/.config/caged/config.json

          SANDBOX_ID=$(caged run \
            --template node-20 \
            --cpus 2 \
            --memory 2048 \
            --repo ${{ github.server_url }}/${{ github.repository }} \
            --budget 3 \
            --env "ANTHROPIC_API_KEY=${{ secrets.ANTHROPIC_API_KEY }}" \
            --json | jq -r '.id')

          caged exec $SANDBOX_ID "npm install && claude 'Run the tests, read the failures, and fix them. Commit your fixes.'"

          # Create PR with fixes
          caged exec $SANDBOX_ID "git push origin HEAD:fix/auto-agent-$(date +%s)"
          caged destroy $SANDBOX_ID --force
```

## PR Review Agent

Run an agent to review every pull request:

```yaml
# .github/workflows/pr-review.yml
name: AI PR Review

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - name: Install Caged CLI
        run: curl -fsSL https://get.caged.dev | sh

      - name: Setup credentials
        run: |
          mkdir -p ~/.config/caged
          echo '{"api_url":"https://api.caged.dev","api_key":"${{ secrets.CAGED_API_KEY }}"}' > ~/.config/caged/config.json

      - name: Run review agent
        run: |
          SANDBOX_ID=$(caged run \
            --template node-20 \
            --cpus 2 \
            --memory 1024 \
            --repo ${{ github.server_url }}/${{ github.repository }} \
            --budget 2 \
            --env "ANTHROPIC_API_KEY=${{ secrets.ANTHROPIC_API_KEY }}" \
            --json | jq -r '.id')

          # Checkout the PR branch
          caged exec $SANDBOX_ID \
            "git fetch origin pull/${{ github.event.number }}/head:pr && git checkout pr"

          # Run review
          REVIEW=$(caged exec $SANDBOX_ID \
            "claude 'Review this branch vs main. List issues, suggestions, and a summary. Output as markdown.'")

          # Post as PR comment
          gh pr comment ${{ github.event.number }} --body "$REVIEW"
          caged destroy $SANDBOX_ID --force
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```

## Tips

<Tip>
**Set tight budgets in CI**: Agent tasks in CI should be predictable. Use `--budget 2-5` to catch runaway loops early.
</Tip>

<Tip>
**Always cleanup**: Use `if: always()` on the destroy step so sandboxes don't leak on workflow failure.
</Tip>

<Tip>
**Use `--json` flag**: Parse sandbox IDs programmatically with `jq` for reliable scripting.
</Tip>