Plan and apply¶
The lifecycle¶
- Open a pull request touching one or more units. The engine plans every impacted unit automatically, one job per unit, in parallel. Only apply is dependency-ordered; a plan reads no other unit's output, so nothing has to wait.
- Read the plan comment. Each unit posts a fresh comment led by a one-line result headline, then the plan output, a change summary, apply instructions, and the validation checks with the gate findings.
- Get the required reviews.
/applyrespects the repo's branch protection. - Apply. Comment
/applyto apply everything in the PR, or/apply -p <label>for one unit. Units apply in dependency order, one at a time. - Merge. Once applied, the
tf-pr-ops / merge-gatecheck turns green and the PR can merge.
Commands¶
| Comment | Effect |
|---|---|
/plan |
Re-plan all impacted units. |
/plan -p <label> |
Plan one unit and its dependents. <label> is the unit path, its name, or its directory. |
/apply |
Apply all impacted units, in dependency order. |
/apply -p <label> |
Apply exactly one unit, no dependents. |
/apply --force |
Apply without the zero-change skip. |
/unlock |
Force-release the unit locks this PR holds. |
/help |
List the commands as a reply on the pull request. |
Opening, updating, or reopening a PR runs /plan automatically. A plan with
changes includes an instructions block telling you exactly which /apply to run.
The impacted set¶
"Impacted" is wider than "has a plan diff", and wider than "the files you edited". A unit is impacted when any of three things is true:
- its own directory changed,
- one of the files it reads changed, or
- it is downstream of an impacted unit through a
dependenciesblock.
The read-files rule is the surprising one. Terragrunt reports every file a unit
pulls in, which includes root.hcl, tenant.hcl, and the subscription.hcl and
region.hcl above it. A one-line change to live/root.hcl therefore impacts
every unit in the tree, because every unit reads it. Expect the full fan-out, and
expect most of those plans to come back as no-ops.
That is the intent. A shared contract change is planned everywhere it lands, not only where someone remembered to look.
Cross-PR unit locks¶
The first pull request to plan a unit owns it until that pull request merges or
closes. A second PR touching the same unit does not plan it: the unit reports a
locked result instead, with a comment naming the owning PR, a failing per-unit
check, and a failing merge gate reading Unit locked by another PR. This stops
a stale plan in one PR from silently reverting what another PR applied.
Locks release five ways:
- Merging or closing the owning PR. This is the normal path; the unlock
caller (
tf-pr-ops-unlock.yml, triggered onpull_request: closed) is what makes it fire. - Commenting
/unlockon the owning PR force-releases its locks, for when a PR is parked but should not block others. The engine confirms with a comment. - Automatically, when the owning PR turns out to be closed. A lock whose holder is already closed or merged (its release event was missed, for example when the head branch was deleted in the same instant as the close) is reclaimed by the next plan that wants the unit, so a stale lock cannot hold a unit hostage.
- On the drift schedule. The nightly drift run sweeps the lock table and releases every lock whose holder PR is closed or merged, so a leak heals without waiting for the unit to be contended. There is no age limit: an open PR's lock stays held however old it is.
- By manual dispatch. The ops workflow's
workflow_dispatchacceptscommand: unlockwith a PR number, for releasing a closed PR's locks by hand when the close event produced no run at all.
Re-running the blocked PR's plan after the lock is gone acquires it.
Ownership lives in an Azure Storage Table (tfprlocks) on the state storage
account, one row per unit; the bootstrap grants the identities access. Locking
skips itself on drift sweeps and manual dispatches, which never own units. A
repository can point locks at a different account with the
TFPR_LOCK_STORAGE_ACCOUNT variable, or turn the feature off with
TFPR_DISABLE_LOCKS set to true. See
Secrets and variables.
The plan comment¶
Each run posts a new comment, so the PR keeps a chronological history of runs. A comment is ordered for scanning:
- A one-line result headline first:
Plan succeeded,Plan failed, orUnit locked by another PR, so the outcome shows at the top instead of below a wall of output. Apply runs get the same headline. - Pre-plan checks, if any pre-plan hooks produced output.
- Plan output in a collapsible block.
- A plan summary table of what will change. Since engine v5.1.0 an
imported resource is a row labelled
import(import,updatewhen the same plan also changes it) and a removed block isforget, so an import-only plan counts as a change and/applylands it. - Instructions with the exact
/applycommands and the/unlockcommand to abandon the changes, expanded, when the plan has changes. - Validation checks from the
post_plangates: policy, security, and cost. This panel is status-aware: collapsed with a check mark when every gate is clean, and auto-expanded with a warning when a gate reports a finding or the run failed, so a policy, security, or cost finding is never hidden.
The per-unit checks on the PR are titled with the change counts (for example
Plan: 2 to add, 1 to change, 0 to destroy), so the checks tab is scannable
without opening runs. When the plan imports something the title leads with
the import count, Plan: 1 to import, 0 to add, 0 to change, 0 to destroy,
and the apply title reads Resources: 1 imported, 0 added, 0 changed, 0
destroyed, the same way Terraform prints them (since engine v5.1.0). When a
new report posts, the unit's previous reports
of the same command are marked as Outdated and get a banner linking to the
report that replaced them; the history stays in the timeline, collapsed,
and only the latest report per command reads at full size, so an apply
report never buries the plan report that was reviewed. Superseded per-unit
check rows of the same command flip to a neutral "Outdated" the same way,
so a failed or refused run's red row does not outlive its successful
retry. Set the TFPR_MINIMIZE_OUTDATED repository variable to false to
opt out of the comment sweep. (Both since engine v3.3.1.)
A push while the PR's apply is still running does not plan on top of it.
The plan and apply jobs of one pull request share a concurrency group, so
the new commit's plan queues until the apply finishes and then plans the new
head. Because GitHub shows the checks of the newest commit only, the running
apply's rows drop out of view the moment you push; the new commit's plan
checks therefore open with the title Waiting for apply #N on <sha>, with a
link to the apply job in their summary, and switch to In progress when
their plan starts. The same applies to a /plan or /apply typed while an
earlier run is still on the runner. (Since engine v6.2.0.)
The apply comment¶
An apply posts a comment of the same shape, and since engine v1.10.0 its Terraform output is filtered down to the real actions. What is hidden is a fixed list: the refresh lines, the ten-second "Still creating..." updates, Terragrunt's log prefixes, its retry chatter, and runs of identical blocks. Every other line passes through, so an error cannot be filtered away, and the verdict comes from the exit code rather than from anything in the text. A resource still running when the apply ended gets a line of its own at the end, so nothing is left mid-create with no outcome.
Plan output is unchanged. It never carried that chatter, so there was nothing to strip.
Since engine v6.0.0 the apply comment carries no validation-checks panel. The
apply re-takes a plan the reviewed-plan guard has just matched to the one the
plan run gated, so the gates are not run again; a one-line note under the
headline says so: "Plan-stage gates not re-run: the plan matches the one
reviewed and gated at <commit>". A repository that wants a gate at apply time
lists it under the apply: stage of steps: (see the
hook and step model).
Comment reactions¶
The engine adds a single 👀 reaction to your /plan or /apply comment, so you
can see the command was picked up without opening the run. That is the only
reaction an accepted command gets. The outcome is carried by the report comment,
the per-unit check, and the merge gate, so a running and a finished reaction only
added noise to the same answer. (Reduced to 👀 alone in engine v1.10.0; before
that a command also collected 🚀, then 🎉 or 👎.)
A refused or unauthorized command still gets 😕, because there nothing else reports the outcome: no run starts, so there is no report comment and no check.
The running and final states appear as a check run (a spinner, then a green
check or red cross) in the PR's Checks section, one per unit, named
tf-pr-ops / <command> (<label>) #<run number> under the org's checks App.
The run number keeps two runs on the same commit apart; when a newer run of
the same unit and command completes, the older row is swept to neutral with
the title Outdated: superseded by #N, so exactly one row per unit and
command stands. Only tf-pr-ops / merge-gate is meant to be a required
check: the per-unit names are not stable across runs, and
tf-pr-ops / approval is informational and must never be required.
An engine-created check has no inline log, so its "View more details" link
matters: since engine v6.1.0 it opens the job that ran the unit (the plan
matrix job, or the single ordered apply job), one click to the live log.
When the engine cannot resolve the job, the link falls back to the run
overview. Since v6.3.0 that log holds the full terraform or terragrunt
output, one foldable group per command (for example terragrunt plan
<label>), so a plan the PR comment truncated can be read in full there.
Since v6.4.0 the check itself also carries a run summary, shown when you
open the row: what triggered the run and who (a push, the comment command,
or a manual dispatch), a table of each resource change as action and
address, and one line per gate (checkov and conftest pass and fail counts,
the infracost monthly total). It never holds plan output or attribute
values; those stay in the PR comment and the job log. Two hidden markers
at the top carry the same facts as data, so tooling can read a run from
the check runs alone without reading comments. A very large change list is
cut with a +N more row to fit GitHub's size limit. When a row is swept to
Outdated, it keeps its summary.
The command gate¶
Before any of that, the engine checks who typed the command. Every command must
come from a comment whose author_association is OWNER, MEMBER, or
COLLABORATOR. Anyone else gets a 😕 reaction and a reply saying the command was
refused.
/apply and /unlock must clear a second check: the commenter's real permission
on the repository, which must be write or admin. Association alone is not
enough for them, because MEMBER means organization member and can include people
with read-only access here. /plan stays on the association filter, so a read
collaborator can still ask for a re-plan on their own PR. (Since engine v1.9.2.)
The approval gate¶
/apply is blocked until the PR meets the repo's required reviews. When it is not
yet approved, the engine posts a comment explaining that apply is blocked, and the
apply job does not run. This uses GitHub's own review decision, so it is configured
entirely in branch protection or rulesets.
A repository that requires no approving reviews at all is a special case: GitHub
reports no review decision, and the engine refuses that too. Such a repository
opts back in with the TFPR_ALLOW_UNREVIEWED_APPLY variable. See
Secrets and variables.
The reviewed-plan guard¶
Apply re-plans each unit just before applying it, so between review and apply the world can move: an out-of-band portal change, drift, or a data source resolving differently. Since engine v3.3.0, every plan report embeds a summary of the plan's resource actions, stamped with the commit it was taken at, and apply compares its fresh plan against that summary per unit. Any difference refuses the apply for that unit: the report shows which resource actions were reviewed but are no longer planned and which are planned but were never reviewed, the per-unit check reads "Plan changed since review", and the merge gate blocks with "Apply blocked: the plan changed since review".
An /apply at a commit that has no plan report is refused the same way, so an
apply on a freshly pushed head cannot ship a plan nobody read. The recovery is
always the same: /plan, review the new plan, /apply again. To apply the
current plan without a re-review, comment /apply --force; the applied report
then says what the force overrode. Force does not bring the plan-stage gates
back either: the operator has read the fresh plan in their place.
Dependency-ordered apply¶
Apply is a single job, not a parallel matrix, so Terragrunt units apply after the units they depend on. If a dependency fails, its dependents are skipped rather than applied against stale state, and the skipped units are reported.
The impacted set is often larger than the set of units with a real diff: a
change to a shared file (like root.hcl) impacts every unit that reads it.
Since engine v3.3.0, a unit whose reviewed plan shows no changes is skipped
outright during apply: no init, no plan, no apply; its check completes green
with "No changes; apply skipped" and no report comment is posted. The one
exception is a unit downstream of a unit that just applied changes: its empty
plan was taken against pre-apply outputs, so it re-plans for real and the
reviewed-plan guard judges the result. /apply --force disables the skip.
The merge gate¶
The tf-pr-ops / merge-gate check is the safety net. It is green
when:
- the PR has no Terraform changes, or
- every unit is a no-op, or
- every impacted unit has been applied, or
- the only Terraform change is the removal of units, which is reported instead.
It is red while changes remain unapplied, when a plan or apply failed, or when an apply was refused because the plan changed since review. It also blocks when the PR changed Terraform but discovery selected no units, so an unapplied net-new unit cannot slip through green. Made a required check, it guarantees the main branch always matches applied infrastructure.
Unattributed Terraform paths¶
Since engine v3.4.0 the same rule covers a wrong-but-nonzero selection. Every
changed Terraform-pattern path must be accounted for by a selected unit, a file
a selected unit reads, an excluded subtree, a removed unit, a
gate_ignore: pattern in projects.yml, or projects.yml itself. Whatever remains blocks every green
verdict with N Terraform path(s) changed outside the planned units, naming
the paths when they fit the check title and always in the gate job's log. So a
PR that changes a shared file no unit reads can no longer plan a subset, go
green, and merge with the rest never planned. Filtered -p runs stay
diagnostics and never write the gate. See
projects.yml for gate_ignore:.
Removed units¶
Deleting a whole unit directory is the one case the checks above cannot see. Discovery walks the pull request's own tree, so a deleted unit is discovered by nothing: never selected, never planned, never destroyed. Its Azure resources and its Terraform state are left behind, and because nothing was selected the gate used to call the pull request "No Terraform changes" and let it merge.
Every run therefore discovers the base branch's tree as well and compares the two
inventories. A unit present on the base branch but missing here is reported, not
blocked. The engine posts a pull request comment naming the unit and carrying the
commands to destroy it by hand, and the merge gate passes with a description like
1 unit removed; destroy by hand, see the PR comment.
Blocking was tried first, in engine v1.9.3, and withdrawn in v1.11.0. The engine cannot destroy a unit it can no longer discover, so a red gate left no path to green except putting the directory back, and a control nobody can satisfy is one people route around.
Renaming a unit reads as one removal plus a new unit at the new path. The
resources stay under the old state key, so a rename is a state move or an import,
never a destroy. Adding a projects.yml exclude for a unit reads the same way and
has the same effect. (Reporting since engine v1.11.0.)
The pass covers the removal and nothing else
The gate passes only when the removal accounts for every changed Terraform path in the pull request. Remove one unit and add another in the same pull request and the gate still blocks, on the one you added. A brand-new unit that discovery cannot see yet looks exactly like a deleted one from the gate's side, and only the removed paths tell the two apart.
A removed-unit check that did not finish grants nothing either. It reports itself incomplete, and an incomplete check cannot excuse a discovery gap it never proved was a removal.
Decommissioning a unit¶
- Delete the unit directory in a pull request.
- Read the comment the engine posts. It names every unit that went and carries the exact commands for the first one.
-
Run the destroy by hand, from a local checkout with access to the state backend and the target subscription:
-
Confirm in the portal that the resources are gone, drop the unit from your working tree again, and merge. The state blob stays where it is: an empty state is harmless, and deleting it is not reversible.
If the resources should keep running, unmanaged by this repository, there is nothing to run. Say so in the pull request description, because after the merge that comment is the only record that the removal was deliberate.
You can also empty the unit first, leaving terragrunt.hcl and an empty
main.tf, and let /apply perform the destroys under the usual review and
approval. That keeps the destroy inside a reviewed plan. Delete the emptied
directory in the next pull request; it reports a removal with nothing left to
destroy, and merges.
Whether the engine should eventually run the destroy itself is still an open decision. Until it is settled the cleanup is yours, and the comment is the record that it was asked for.
A filtered run (/plan -p or /apply -p) never moves the gate, in either
direction. It looks at one unit, so it cannot judge the whole PR; the verdict
of the last full run stands until the next full plan or apply. (Since engine
v1.9.1.)