Bootstrap a customer¶
The bootstrap is a plain Terraform root module in nrit-alz-bootstrap. An operator
runs it once per customer, from a workstation, against the customer tenant. There is
no CI for it and no pipeline to trigger. It stands up everything the customer
repository needs before any workflow can run: the state backend, the deploy
identities and their OIDC federation, the repository itself, and, on the production
posture, the self-hosted runners.
This page is the procedure. For the full list of what it creates and every input it takes, see Bootstrap reference.
Before you start¶
You need three things: an Azure session, GitHub tokens, and rights in both.
Azure¶
Sign in to the customer tenant and select the management subscription. That subscription holds the state, the identities, and the runners.
Your account needs:
- rights to create resource groups and resources in the management subscription,
- rights to create role assignments at the management group scope you are targeting (the plan identity gets Reader there, the apply identity gets Owner), which in practice means User Access Administrator or Owner at that scope,
- rights to read your own directory object, which
scripts/create-backend.shuses to grant you Storage Blob Data Contributor on the state account.
GitHub¶
A personal access token with repo and admin:org scope. The admin:org scope is
needed because the bootstrap creates the approver team.
GITHUB_TOKEN authenticates the github provider. Some paths need the token as a
real Terraform value instead, so export TF_VAR_github_token when either applies:
| Variable | When you need it |
|---|---|
GITHUB_TOKEN or TF_VAR_github_token |
Always. Authenticates the github provider. |
TF_VAR_github_runners_token |
self_hosted_private only. Registers the runners. Can be the same PAT. Falls back to github_token if unset. |
TF_VAR_github_token (not just GITHUB_TOKEN) |
Building a custom runner image. The ACR build task clones the context repo with a real Terraform value. TF_VAR_runner_image_context_token overrides it. |
TF_VAR_catalog_app_private_key |
Whenever the tfvars file sets catalog_app_client_id. |
The GitHub App private key¶
The GitHub App private key is what authenticates the private engine checkout for the
PR-ops engine (nrit-tf-pr-ops). The CATALOG_APP_* names are historical, kept from
a module catalog that no longer exists. See
Secrets and variables for what the engine does with them.
Omitting the key destroys the secret
catalog_app_private_key defaults to null, and the CATALOG_APP_PRIVATE_KEY
repository secret is managed only when it is set. Run the bootstrap without it on
a customer that has the secret, and Terraform destroys the secret. The engine
checkout then fails on the next run. Always export it when
catalog_app_client_id is set in the tfvars file.
Secrets go in the environment, never in tfvars
The tfvars files are committed. The PAT, the runners PAT, and the App private key are only ever supplied through environment variables. There are no long-lived Azure secrets at all: storage account keys are disabled and all pipeline auth is OIDC.
Run it¶
1. Create the customer var file¶
Fill in the tenant id, the management subscription id, the repository name, the
approvers, and the network posture. Set network_posture = "self_hosted_private" for
production. Set required_status_checks = ["tf-pr-ops / merge-gate"] unless you have
a reason not to: the default is empty, which lets a pull request merge with unapplied
changes. Every input is listed in
Bootstrap reference.
2. Create the state backend¶
The bootstrap's own state lives in a dedicated storage account that create-backend.sh
owns out of band. A config must not manage the account that stores its own state, so
this account is not in the Terraform.
The script is idempotent. Each run:
- ensures the resource group
rg-cicd-bootstrap-<location_short>and the storage account (name derived deterministically from the subscription id, so no shared state is needed to compute it), - disables shared keys, enforces TLS 1.2, and sets the account default-deny,
- enables blob versioning and seven-day soft delete,
- resets the IP firewall to your current public IP, removing every other entry,
- ensures the
tfstatecontainer through the ARM control plane, - grants you Storage Blob Data Contributor,
- prints the exact
terraform initcommand to run next.
Re-run the script whenever your public IP changes
The firewall allows exactly one operator IP. Change network, or get a new address
from your ISP, and every terraform command fails on the backend. Re-run
scripts/create-backend.sh before any terraform command; it resets the allowlist
to your current IP.
3. Initialise Terraform¶
The backend block carries no values, because backend blocks cannot use variables. Use
the -backend-config lines the script printed.
terraform init \
-backend-config="resource_group_name=rg-cicd-bootstrap-<loc>" \
-backend-config="storage_account_name=<printed by the script>" \
-backend-config="container_name=tfstate" \
-backend-config="key=bootstrap.tfstate"
First run only
The script grants you Storage Blob Data Contributor on the account it just created.
That role assignment can take about a minute to propagate. If terraform init
fails with an authorisation error on the very first run, wait and retry.
4. Plan and apply¶
terraform plan -var-file=customers/<customer>.tfvars
terraform apply -var-file=customers/<customer>.tfvars
Read the plan. On the production posture it is large, because the AVM runner module creates a network, a registry, and the runner compute. Check the known permadiff below before you apply.
Re-running the bootstrap¶
Re-running is normal. You re-run to add a capability to an existing customer, to change a decision variable, or to recover from a partial failure.
- Export the same environment variables as the first run, including
TF_VAR_catalog_app_private_key. Without it theCATALOG_APP_PRIVATE_KEYsecret is destroyed. - Re-run
scripts/create-backend.sh -f customers/<customer>.tfvars, in case your IP changed. terraform planandterraform applywith the same var file.
State is read from the remote backend, so a re-run does not depend on a local
terraform.tfstate file or on the machine that did the first run.
The remote state must be preserved
A clean apply against empty state tries to create a GitHub repository that already
exists, and fails. If bootstrap-managed resources were deleted out of band,
terraform apply treats that as drift and recreates them, while the existing
repository and its Action variables are updated in place.
Rolling out a custom runner image¶
By default the AVM module builds its own runner image. That image has no python3,
checkov, or conftest, so the engine's policy and scan hooks cannot run on a
self-hosted runner. Set runner_image_commit to build the vendored image in
runner-image/github-runner-aca instead, which is the upstream AVM image plus one
tools layer.
export TF_VAR_github_token=<pat> # the ACR build task needs a real Terraform value
terraform apply -target='module.runners' -var-file=customers/<customer>.tfvars
Use a commit SHA, not a branch. The image tag is content-addressed by the SHA
(github-runner:nrit-<first 7 chars>), so a Dockerfile change produces a new tag and
the Container App Job redeploys. A branch ref produces a stable tag and nothing rolls
out.
Known permadiff: the ACR bypass revert¶
Never apply the ACR bypass revert
Every plan on the self_hosted_private posture shows
azurerm_container_registry.this wanting to set
network_rule_bypass_for_tasks_enabled = true -> false. Do not apply it.
The flag is owned by an azapi_update_resource patch inside the AVM runners
module. The wrapped azurerm resource does not know about the patch and tries to
revert it. Applying the revert breaks the next runner-image build, because the
registry is private and the build task needs the bypass. The azapi patch does not
self-heal on drift.
If it is applied by accident, restore it:
Use -target for runner changes, or read the plan carefully, so the revert never
slips in with an unrelated apply.
What the bootstrap does not do¶
Four things are manual after the apply. None of them are managed by the bootstrap, so a re-run will not create or restore them.
| Step | Where | Why it is manual |
|---|---|---|
Set the INFRACOST_API_KEY secret on the customer repository |
Repository secrets | The key is an account-level credential, not a per-customer value. Until it is set, the cost gate notes it and skips. |
Make nrit-tf-pr-ops organisation-accessible for Actions |
Engine repository settings | Without it the customer repository cannot call the engine's reusable workflows. |
Set TF_PR_OPS_ALLOW_UNREVIEWED_APPLY to true |
Repository variables | Only for a single-writer organisation that set required_approving_review_count = 0. GitHub then returns an empty review decision and the engine refuses to apply on it. |
Add CODEOWNERS and LICENSE to the generated repository |
Customer repository | The bootstrap does no templating and writes no files. Everything in the generated repository is a verbatim copy of the template. |
See Secrets and variables for the full list of what the repository reads and who sets each value.
After the bootstrap¶
The customer repository now has its backend, identities, environments, and merge gate wired. Next, pin the engine version in the two caller workflows and fill in the handful of placeholder values in the tree. The Quickstart walks that end to end.
If something fails, Troubleshooting covers the common bootstrap errors.
Next: Bootstrap reference.