GitHub Action · canary analysis

Is this deploy safe?
Let the pipeline decide.

deploy-gate compares a canary's real metrics with the version already serving traffic and returns promote rollback or extend, with a report that says why. Missing or thin data never promotes.

Why

Deploys are cheap, judgment is not

Most teams ship a canary, open a dashboard and decide by eye. That depends on who is watching, how tired they are and which graph they happened to open.

Fixed thresholds lie

"Alert if errors exceed 1%" fires at peak traffic and stays quiet at night. Comparing the canary with the current version over the same minutes cancels those patterns out.

Noise is real

With little traffic, one bad request looks like a 5% error rate. A check fails only when the canary is worse by more than the tolerance you chose, with confidence, so ordinary noise never rolls anything back.

"Ready" is a gate the system passes, not a feeling.

How it works

Select a step to see what happens there.

Try it

Synthetic traffic, real decision code: this page runs the same src/core the action ships. Pick a scenario or move the sliders.

Check Verdict p-value Reason

baseline p99 canary p99 (per minute)

How to use it

  1. Give the workflow read-only access to CloudWatch

    Create an IAM role GitHub can assume through OIDC, allowed only cloudwatch:GetMetricData. No long-lived keys; the action never writes to AWS.

  2. Describe your checks

    # .github/deploy-gate.yml
    source: cloudwatch
    maxDurationMinutes: 30
    checks:
      - { name: errors, kind: error-rate, maxAbsoluteIncrease: 0.005 }
      - { name: latency-p99, kind: latency, statistic: p99, maxRelativeIncrease: 0.10 }
    groups:
      canary:   { dimensions: { TargetGroup: targetgroup/app-canary/abc, LoadBalancer: app/main/123 } }
      baseline: { dimensions: { TargetGroup: targetgroup/app-stable/def, LoadBalancer: app/main/123 } }
  3. Call it after the canary deploy, and act on the decision

    - id: gate
      uses: pablomagnaghi/deploy-gate@v1
      with:
        canary-started-at: ${{ steps.canary.outputs.started-at }}
        fail-on-rollback: false
    - if: steps.gate.outputs.decision == 'promote'
      run: ./scripts/shift-traffic.sh 100
    - if: steps.gate.outputs.decision == 'rollback'
      run: ./scripts/shift-traffic.sh 0

    On extend, wait and call it again: the window grows until the checks have enough data or the time limit rolls back. The examples/ folder wires this up for ALB weights, CodeDeploy and Lambda aliases.

  4. Read the report

    Every run writes a job summary: the decision, the reason, and one row per check with the numbers and the p-value, so anyone can verify it by hand.

Principles