Drift detection¶

A scheduled workflow checks whether the live tenant still matches the code, and turns any drift into GitHub Issues. There is no PR to comment on, so the issue is the signal.

What runs¶

The drift workflow runs daily (early morning UTC) and on demand. It has three jobs:

resolve (hosted runner): selects every unit for the sweep. An optional drift: block in projects.yml can scope or disable it; by default all units are swept. Its per-job check appears as drift / engine / resolve: the caller workflow's name, then the caller's job id, then the engine's job name.
```
drift:
  enabled: true          # false disables the sweep entirely
  include:               # optional, repo-relative globs
    - live/_foundation/**
  exclude:
    - live/sandbox/**
```
2. drift-plan (self-hosted runner, plan environment): plans each unit against the deployed tenant, read-only. It uses the same read-only Reader identity as PR plans, so it can never change the tenant. A non-empty plan is drift, which is expected, so it never fails the job. It runs the same gates as a PR plan. 3. report (hosted runner): turns the results into issues.

The issues¶

The report job opens one issue per drifted unit, titled Drift: <label> with a drift label. It:

opens or refreshes the issue when a unit has changes or errored,
comments and closes the issue when the unit comes back clean.

So the set of open drift issues is always the current list of units that no longer match their code.

An issue body is the same run report a PR plan posts: the plan output in a code fence, the change summary table, and the gate findings. An issue for a unit that errored (rather than drifted) carries the error output instead of a plan, so read the headline: changes means drift, error means the plan itself failed and needs troubleshooting.

The sweep is per unit

Drift planning does not pull in downstream dependents the way a PR plan does. Each unit is planned on its own, so drift in a shared unit shows up only as that unit's issue, not as issues on everything that depends on it.

Why it is split across runners¶

Only the plan needs Terraform state, which sits behind a private endpoint, so it must run on the self-hosted runner. Issue management needs only the GitHub API, so it runs on a hosted runner. The drift plan shares a per-unit concurrency group with PR plans, so a drift plan and a PR plan for the same unit never contend for the state lock.

Acting on drift¶

Drift itself never opens a pull request. To remediate, an operator opens a PR that touches the drifted unit, which plans it the normal way. Then either apply to bring the tenant back in line, or update the code to match an intended change. Applying closes the drift issue on the next sweep.

The runbook has a triage guide by cause: portal changes, policy remediation side effects, provider upgrades, and AMBA tag stripping each look different in the plan.

The first sweep is the noisy one

On a brownfield tenant the first sweep can open many issues at once. Triage them by unit, not one resource at a time: most issues share one cause, and a single PR that reconciles the shared cause closes them together on the next sweep.