Skip to content

Bootstrap a customer

The bootstrap is a plain Terraform root module in nrit-alz-bootstrap. An operator runs it once per customer, from a workstation, against the customer tenant. There is no CI for it and no pipeline to trigger. It stands up everything the customer repository needs before any workflow can run: the state backend, the deploy identities and their OIDC federation, the repository itself, and, on the production posture, the self-hosted runners.

This page is the procedure. For the full list of what it creates and every input it takes, see Bootstrap reference.

Before you start

You need three things: an Azure session, GitHub tokens, and rights in both.

Azure

Sign in to the customer tenant and select the management subscription. That subscription holds the state, the identities, and the runners.

az login --tenant <customer-tenant-id>
az account set --subscription <management-subscription-id>

Your account needs:

  • rights to create resource groups and resources in the management subscription,
  • rights to create role assignments at the management group scope you are targeting (the plan identity gets Reader there, the apply identity gets Owner), which in practice means User Access Administrator or Owner at that scope,
  • rights to read your own directory object, which scripts/create-backend.sh uses to grant you Storage Blob Data Contributor on the state account.

GitHub

A personal access token with repo and admin:org scope. The admin:org scope is needed because the bootstrap creates the approver team.

export GITHUB_TOKEN=<pat>

GITHUB_TOKEN authenticates the github provider. Some paths need the token as a real Terraform value instead, so export TF_VAR_github_token when either applies:

Variable When you need it
GITHUB_TOKEN or TF_VAR_github_token Always. Authenticates the github provider.
TF_VAR_github_runners_token self_hosted_private only. Registers the runners. Can be the same PAT. Falls back to github_token if unset.
TF_VAR_github_token (not just GITHUB_TOKEN) Building a custom runner image. The ACR build task clones the context repo with a real Terraform value. TF_VAR_runner_image_context_token overrides it.
TF_VAR_catalog_app_private_key Whenever the tfvars file sets catalog_app_client_id.

The GitHub App private key

The GitHub App private key is what authenticates the private engine checkout for the PR-ops engine (nrit-tf-pr-ops). The CATALOG_APP_* names are historical, kept from a module catalog that no longer exists. See Secrets and variables for what the engine does with them.

export TF_VAR_catalog_app_private_key="$(cat /path/to/app-private-key.pem)"

Omitting the key destroys the secret

catalog_app_private_key defaults to null, and the CATALOG_APP_PRIVATE_KEY repository secret is managed only when it is set. Run the bootstrap without it on a customer that has the secret, and Terraform destroys the secret. The engine checkout then fails on the next run. Always export it when catalog_app_client_id is set in the tfvars file.

Secrets go in the environment, never in tfvars

The tfvars files are committed. The PAT, the runners PAT, and the App private key are only ever supplied through environment variables. There are no long-lived Azure secrets at all: storage account keys are disabled and all pipeline auth is OIDC.

Run it

1. Create the customer var file

cd nrit-alz-bootstrap
cp customers/_example.tfvars customers/<customer>.tfvars

Fill in the tenant id, the management subscription id, the repository name, the approvers, and the network posture. Set network_posture = "self_hosted_private" for production. Set required_status_checks = ["tf-pr-ops / merge-gate"] unless you have a reason not to: the default is empty, which lets a pull request merge with unapplied changes. Every input is listed in Bootstrap reference.

2. Create the state backend

The bootstrap's own state lives in a dedicated storage account that create-backend.sh owns out of band. A config must not manage the account that stores its own state, so this account is not in the Terraform.

scripts/create-backend.sh -f customers/<customer>.tfvars

The script is idempotent. Each run:

  • ensures the resource group rg-cicd-bootstrap-<location_short> and the storage account (name derived deterministically from the subscription id, so no shared state is needed to compute it),
  • disables shared keys, enforces TLS 1.2, and sets the account default-deny,
  • enables blob versioning and seven-day soft delete,
  • resets the IP firewall to your current public IP, removing every other entry,
  • ensures the tfstate container through the ARM control plane,
  • grants you Storage Blob Data Contributor,
  • prints the exact terraform init command to run next.

Re-run the script whenever your public IP changes

The firewall allows exactly one operator IP. Change network, or get a new address from your ISP, and every terraform command fails on the backend. Re-run scripts/create-backend.sh before any terraform command; it resets the allowlist to your current IP.

3. Initialise Terraform

The backend block carries no values, because backend blocks cannot use variables. Use the -backend-config lines the script printed.

terraform init \
  -backend-config="resource_group_name=rg-cicd-bootstrap-<loc>" \
  -backend-config="storage_account_name=<printed by the script>" \
  -backend-config="container_name=tfstate" \
  -backend-config="key=bootstrap.tfstate"

First run only

The script grants you Storage Blob Data Contributor on the account it just created. That role assignment can take about a minute to propagate. If terraform init fails with an authorisation error on the very first run, wait and retry.

4. Plan and apply

terraform plan  -var-file=customers/<customer>.tfvars
terraform apply -var-file=customers/<customer>.tfvars

Read the plan. On the production posture it is large, because the AVM runner module creates a network, a registry, and the runner compute. Check the known permadiff below before you apply.

Re-running the bootstrap

Re-running is normal. You re-run to add a capability to an existing customer, to change a decision variable, or to recover from a partial failure.

  1. Export the same environment variables as the first run, including TF_VAR_catalog_app_private_key. Without it the CATALOG_APP_PRIVATE_KEY secret is destroyed.
  2. Re-run scripts/create-backend.sh -f customers/<customer>.tfvars, in case your IP changed.
  3. terraform plan and terraform apply with the same var file.

State is read from the remote backend, so a re-run does not depend on a local terraform.tfstate file or on the machine that did the first run.

The remote state must be preserved

A clean apply against empty state tries to create a GitHub repository that already exists, and fails. If bootstrap-managed resources were deleted out of band, terraform apply treats that as drift and recreates them, while the existing repository and its Action variables are updated in place.

Rolling out a custom runner image

By default the AVM module builds its own runner image. That image has no python3, checkov, or conftest, so the engine's policy and scan hooks cannot run on a self-hosted runner. Set runner_image_commit to build the vendored image in runner-image/github-runner-aca instead, which is the upstream AVM image plus one tools layer.

# in customers/<customer>.tfvars
runner_image_commit = "<commit sha of nrit-alz-bootstrap>"
export TF_VAR_github_token=<pat>   # the ACR build task needs a real Terraform value
terraform apply -target='module.runners' -var-file=customers/<customer>.tfvars

Use a commit SHA, not a branch. The image tag is content-addressed by the SHA (github-runner:nrit-<first 7 chars>), so a Dockerfile change produces a new tag and the Container App Job redeploys. A branch ref produces a stable tag and nothing rolls out.

Known permadiff: the ACR bypass revert

Never apply the ACR bypass revert

Every plan on the self_hosted_private posture shows azurerm_container_registry.this wanting to set network_rule_bypass_for_tasks_enabled = true -> false. Do not apply it.

The flag is owned by an azapi_update_resource patch inside the AVM runners module. The wrapped azurerm resource does not know about the patch and tries to revert it. Applying the revert breaks the next runner-image build, because the registry is private and the build task needs the bypass. The azapi patch does not self-heal on drift.

If it is applied by accident, restore it:

az rest --method patch \
  --url "https://management.azure.com/<acr-resource-id>?api-version=2025-05-01-preview" \
  --body '{"properties":{"networkRuleBypassAllowedForTasks":true}}'

Use -target for runner changes, or read the plan carefully, so the revert never slips in with an unrelated apply.

What the bootstrap does not do

Four things are manual after the apply. None of them are managed by the bootstrap, so a re-run will not create or restore them.

Step Where Why it is manual
Set the INFRACOST_API_KEY secret on the customer repository Repository secrets The key is an account-level credential, not a per-customer value. Until it is set, the cost gate notes it and skips.
Make nrit-tf-pr-ops organisation-accessible for Actions Engine repository settings Without it the customer repository cannot call the engine's reusable workflows.
Set TF_PR_OPS_ALLOW_UNREVIEWED_APPLY to true Repository variables Only for a single-writer organisation that set required_approving_review_count = 0. GitHub then returns an empty review decision and the engine refuses to apply on it.
Add CODEOWNERS and LICENSE to the generated repository Customer repository The bootstrap does no templating and writes no files. Everything in the generated repository is a verbatim copy of the template.

See Secrets and variables for the full list of what the repository reads and who sets each value.

After the bootstrap

The customer repository now has its backend, identities, environments, and merge gate wired. Next, pin the engine version in the two caller workflows and fill in the handful of placeholder values in the tree. The Quickstart walks that end to end.

If something fails, Troubleshooting covers the common bootstrap errors.

Next: Bootstrap reference.