Skip to content
GitHub
Start a 14-day trial

Record your deploys

When an incident opens, the first question is what changed? Record your deploys and I answer it on the incident itself: its Recent changes list shows the deploys that shipped in the 6 hours before it, newest first, and the Slack alert carries the same list.

A deploy you record also opens a 10-minute warm-up for that service. A freshly started service is often slow for its first minute or two, so a latency spike inside the warm-up is raised as a warning instead of paging you as critical. Error rates are never held back. If the slowness outlasts the warm-up, the incident escalates and pages as usual.

Detected or recorded: either is enough, both is best

Section titled “Detected or recorded: either is enough, both is best”

Detected, with no setup. If your services report a version — service.version, as Connect your app recommends — I watch for a new version taking over and record the deploy myself, marked (detected from service.version). I read the version from service.version, else version, deployment.version or vcs.ref.head.revision. A deploy counts once the new version has carried at least half of the service’s traffic for 10 minutes, after another version held it, so a canary is recorded only when it takes over. A detected deploy is dated from the new version’s first traffic and shows up 10 to about 70 minutes after the rollout; the incidents that open from then on list it.

Recorded from your CI. One request from your deploy pipeline, sent as the rollout starts. A recorded deploy:

  • carries the exact time it shipped;
  • is listed on incidents that open right after the rollout, before detection would catch it;
  • opens the warm-up as the rollout starts;
  • covers rollbacks: a rollback you record is listed like any other deploy, even to a version I have seen before. Detection records each version of a service only once.

Running both doesn’t list a deploy twice: when a recorded deploy and a detected one have the same service and version, and the time you record is within 15 minutes of the new version’s first traffic, you see it once, as the deploy you recorded. Any service your pipeline doesn’t record is still covered by detection, as long as it reports a version.

  1. Open Admin & Governance → Settings → App Onboarding.
  2. Click Generate token, choose Record deploys in the dialog, then click Generate token.
  3. Copy the token now — it is shown only once. Store it as a secret in your CI, for example ATRIM_DEPLOY_TOKEN.

A deploy token only records changes such as deploys: it can’t send telemetry or read your data. Telemetry goes with an ingest token, the default Send telemetry purpose; the Purpose column of the token list shows which is which. Because each recorded deploy opens the warm-up, keep the token secret like any other. If it leaks, revoke it in App Onboarding with Revoke immediately. Only a workspace owner or admin creates tokens.

Run this from your pipeline as the rollout starts, so the warm-up covers the rollout itself: just before your deploy command, or right after it if the command returns without waiting for the rollout (a plain kubectl apply). A command that waits, such as helm upgrade --wait or kubectl rollout status, finishes too late.

curl -sS --fail-with-body -X POST https://api.atrim.ai/api/changes \
  -H "x-api-key: $ATRIM_DEPLOY_TOKEN" \
  -H "content-type: application/json" \
  -d @- <<EOF
{
  "kind": "deploy",
  "occurredAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
  "scope": { "serviceNamespace": "shop", "serviceName": "cart" },
  "after": { "version": "1.5.0" },
  "actor": "ci"
}
EOF

Fill in three values so the deploy lands on the right service:

Field Set it to
scope.serviceName The service’s service.name, exactly as your telemetry reports it.
scope.serviceNamespace The service’s service.namespace, exactly as your telemetry reports it: an incident lists the deploys in its own namespace, so a value your telemetry doesn’t report keeps the deploy off your incidents. If your services don’t set one, leave it out or empty, and the deploy is listed on incidents in every namespace.
after.version The version the new release reports as service.version. When it matches, a detected deploy of the same release, within 15 minutes, folds into this one.

occurredAt is the deploy time, in UTC; the sample stamps the current time. actor names who or what shipped it. Add "before": { "version": "1.4.2" } and the incident reads version: 1.4.2 → 1.5.0. Deploying several services at once? Send one request per service.

The API answers 201 for a new deploy. The identical request, with the same occurredAt, sent again answers 200 and records nothing new; a re-run of the job stamps a new time and records the deploy again. 401 means the token is missing, revoked, or an ingest token rather than a deploy token; 400 says what it couldn’t accept.

Add a step just before your deploy step, or right after it if that step doesn’t wait for the rollout. Save the token as a repository secret named ATRIM_DEPLOY_TOKEN first.

      - name: Record the deploy in atrim.ai
        continue-on-error: true
        env:
          ATRIM_DEPLOY_TOKEN: ${{ secrets.ATRIM_DEPLOY_TOKEN }}
          SERVICE_NAMESPACE: shop
          SERVICE_NAME: cart
          VERSION: ${{ github.sha }}
        run: |
          curl -sS --fail-with-body -X POST https://api.atrim.ai/api/changes \
            -H "x-api-key: $ATRIM_DEPLOY_TOKEN" \
            -H "content-type: application/json" \
            -d @- <<EOF
          {
            "kind": "deploy",
            "occurredAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
            "scope": { "serviceNamespace": "$SERVICE_NAMESPACE", "serviceName": "$SERVICE_NAME" },
            "after": { "version": "$VERSION" },
            "actor": "$GITHUB_ACTOR",
            "links": [{ "type": "external", "url": "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" }]
          }
          EOF

Set VERSION to whatever your build stamps into service.version — the commit SHA here, or a tag such as ${{ github.ref_name }}. The link puts the workflow run one click away on the incident. continue-on-error keeps a failed record from turning a good deploy red. Deploying several services? Repeat the step once per service.

  • Incidents — the Recent changes list on an incident shows up to 10 changes from the 6 hours before it opened, newest first, with each deploy’s service, version and links. It covers every service in the incident’s namespace, not only the one that alerted.
  • Slack — firing and acknowledged incident alerts carry the same list.
  • Webhooks — the payload’s context.recent_changes array, one entry per change.

The list is taken when the incident opens and again if it reopens, so it always describes the current incident. Questions about recording your deploys? Write to us at hello@atrim.ai — we’ll look at it with you.