Use this procedure when a reviewed job needs to run outside an untrusted application Maintainer's authority. A trusted GitLab group Owner creates a private subgroup, protects its only project's default branch, registers a paused group runner, and administers the local Docker host. The job performs a normal GitLab checkout and artifact upload while its task container can reach GitLab HTTPS only through a small CONNECT proxy.
This procedure runs one finite job on a shared local Docker host. It uses one Owner-authorized commit. It doesn't make the host suitable for arbitrary hostile code, prove kernel separation, establish packet-level egress absence, deliver secrets, or stop an application Maintainer from selecting another runner in a different application project. Use a dedicated host or stronger executor when those properties matter. GitLab's runner security guidance explains why the Docker socket and shared hosts need this trust boundary.
The example targets Linux ARM64 with Docker Engine, Bash, glab, jq, Git, and Python 3. Repin every image for another architecture or Runner version. The helper version must match the Runner version, as described in GitLab's advanced Runner configuration.
Separate the administrators
Start by separating who can change the application from who can change its execution environment:
- A trusted parent-group Owner creates the subgroup, project, protected branch, and group runner. The Owner also controls the Docker host and runner token.
- Application Maintainers remain ordinary, untrusted project participants. Don't give them Owner access to this subgroup or access to the Docker host, proxy configuration, runner token, or local
config.toml. - The protected job uses an Owner-authorized commit on
main. This procedure doesn't admit application-Maintainer-authored CI changes.
Choose a parent group where the Owner has already reviewed inherited members, CI/CD variables, runners, security policies, shares, and custom roles. A private subgroup inherits authority from its ancestors, so a new path alone isn't a trust boundary.
The main path is easiest to follow in one trusted Bash session, but cleanup must not depend on that shell surviving. Choose a durable, access-controlled directory outside the guide checkout for the recovery record and Runner token file. The recovery record contains identifiers and fixed names, not the token. Set the local placeholders in the Owner's authenticated glab session:
set -euo pipefail
export GSS_PARENT_GROUP_ID="<trusted-parent-group-id>"
export GSS_OWNER_USER_ID="<trusted-owner-user-id>"
export GSS_SUBGROUP_PATH="dedicated-runner-practice"
export GSS_PROJECT_PATH="runner-boundary"
export GSS_RUNNER_TAG="gss-isolated-arm64"
export GSS_REVIEW_BY="<yyyy-mm-dd>"
export GSS_GUIDE_CHECKOUT="<absolute-path-to-reviewed-guide-checkout>"
export GSS_RECOVERY_DIR="<absolute-private-path>/gss-dedicated-runner-recovery"
export GSS_DOCKER_CONTEXT="desktop-linux"
gss_docker() { docker --context "${GSS_DOCKER_CONTEXT}" "$@"; }
test "$(glab api --hostname gitlab.com user | jq -er '.id')" \
= "${GSS_OWNER_USER_ID}"
docker context inspect "${GSS_DOCKER_CONTEXT}" >/dev/null
gss_docker info >/dev/null
test "${GSS_RECOVERY_DIR}" = "$(cd "$(dirname "${GSS_RECOVERY_DIR}")" && pwd)/$(basename "${GSS_RECOVERY_DIR}")"
test ! -e "${GSS_RECOVERY_DIR}"
umask 077
mkdir -p "${GSS_RECOVERY_DIR}/runner-config"
export GSS_PRIVATE_DIR="${GSS_RECOVERY_DIR}/runner-config"
export GSS_RECOVERY_FILE="${GSS_RECOVERY_DIR}/recovery.json"
GSS_PARENT_GROUP_JSON="$(glab api --hostname gitlab.com \
"groups/${GSS_PARENT_GROUP_ID}")"
GSS_PARENT_FULL_PATH="$(jq -er '.full_path' <<<"${GSS_PARENT_GROUP_JSON}")"
GSS_EXISTING_GROUPS_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"groups/${GSS_PARENT_GROUP_ID}/subgroups?per_page=100" \
| jq -s 'add' >"${GSS_EXISTING_GROUPS_FILE}"
test "$(jq --arg path "${GSS_PARENT_FULL_PATH}/${GSS_SUBGROUP_PATH}" \
'[.[] | select(.full_path == $path)] | length' \
"${GSS_EXISTING_GROUPS_FILE}")" -eq 0
rm -f "${GSS_EXISTING_GROUPS_FILE}"
jq -n --argjson parent_group_id "${GSS_PARENT_GROUP_ID}" \
--arg parent_full_path "${GSS_PARENT_FULL_PATH}" \
--arg owner_user_id "${GSS_OWNER_USER_ID}" \
--arg subgroup_path "${GSS_SUBGROUP_PATH}" \
--arg project_path "${GSS_PROJECT_PATH}" \
--arg runner_tag "${GSS_RUNNER_TAG}" \
--arg review_by "${GSS_REVIEW_BY}" \
--arg docker_context "${GSS_DOCKER_CONTEXT}" \
--arg private_dir "${GSS_PRIVATE_DIR}" \
'{parent_group_id: $parent_group_id, parent_full_path: $parent_full_path,
owner_user_id: $owner_user_id, subgroup_path: $subgroup_path,
project_path: $project_path, runner_tag: $runner_tag, review_by: $review_by,
group_path_absent_before_create: true,
docker_context: $docker_context, private_dir: $private_dir}' \
>"${GSS_RECOVERY_FILE}"
unset GSS_PARENT_GROUP_JSON GSS_PARENT_FULL_PATH
gss_record() {
key="$1"
value="$2"
update="$(mktemp "${GSS_RECOVERY_DIR}/recovery.XXXXXX")"
jq --arg key "${key}" --arg value "${value}" \
'. + {($key): $value}' "${GSS_RECOVERY_FILE}" >"${update}"
mv "${update}" "${GSS_RECOVERY_FILE}"
}
gss_docker volume ls -q | sort >"${GSS_RECOVERY_DIR}/baseline-volumes.txt"
This record exists before the first GitLab or Docker mutation. Keep the whole directory private: recovery.json is non-secret, while runner-config will contain the one-time runner token. An existing directory means an earlier run still needs reconciliation; use the recovery procedure below instead of overwriting it.
Create one private project and protect main
The subgroup temporarily permits project creation only long enough for the Owner to create its single project. Resetting the setting before runner registration avoids leaving a creation window while execution authority exists.
set -euo pipefail
GSS_GROUP_JSON="$(glab api --hostname gitlab.com --method POST groups \
--raw-field "parent_id=${GSS_PARENT_GROUP_ID}" \
--raw-field "name=${GSS_SUBGROUP_PATH}" \
--raw-field "path=${GSS_SUBGROUP_PATH}" \
--raw-field visibility=private \
--raw-field project_creation_level=maintainer)"
GSS_GROUP_ID="$(jq -er '.id' <<<"${GSS_GROUP_JSON}")"
GSS_GROUP_FULL_PATH="$(jq -er '.full_path' <<<"${GSS_GROUP_JSON}")"
gss_record group_id "${GSS_GROUP_ID}"
gss_record group_full_path "${GSS_GROUP_FULL_PATH}"
GSS_PROJECT_JSON="$(glab api --hostname gitlab.com --method POST projects \
--raw-field "namespace_id=${GSS_GROUP_ID}" \
--raw-field "name=${GSS_PROJECT_PATH}" \
--raw-field "path=${GSS_PROJECT_PATH}" \
--raw-field visibility=private \
--raw-field initialize_with_readme=true \
--raw-field default_branch=main \
--raw-field auto_devops_enabled=false)"
GSS_PROJECT_ID="$(jq -er '.id' <<<"${GSS_PROJECT_JSON}")"
GSS_PROJECT_HTTP_URL="$(jq -er '.http_url_to_repo' <<<"${GSS_PROJECT_JSON}")"
gss_record project_id "${GSS_PROJECT_ID}"
gss_record project_http_url "${GSS_PROJECT_HTTP_URL}"
glab api --hostname gitlab.com --method PUT "groups/${GSS_GROUP_ID}" \
--raw-field project_creation_level=noone >/dev/null
test "$(glab api --hostname gitlab.com "groups/${GSS_GROUP_ID}" \
| jq -er '.project_creation_level')" = noone
Before continuing, paginate the subgroup's projects, descendants, shares, and members. The inventory should contain the expected single project and no unexpected descendant or share. Keep inherited Maintainers visible in that inventory; removing their access merely to make the checks pass would hide the authority you need to assess.
set -euo pipefail
GSS_PROJECTS_FILE="$(mktemp)"
GSS_DESCENDANTS_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"groups/${GSS_GROUP_ID}/projects?include_subgroups=true&with_shared=false&per_page=100" \
| jq -s 'add' >"${GSS_PROJECTS_FILE}"
glab api --hostname gitlab.com --paginate \
"groups/${GSS_GROUP_ID}/descendant_groups?per_page=100" \
| jq -s 'add' >"${GSS_DESCENDANTS_FILE}"
jq -e --argjson project "${GSS_PROJECT_ID}" '
length == 1 and .[0].id == $project and .[0].visibility == "private"' \
"${GSS_PROJECTS_FILE}" >/dev/null
test "$(jq 'length' "${GSS_DESCENDANTS_FILE}")" -eq 0
glab api --hostname gitlab.com "groups/${GSS_GROUP_ID}" \
| jq -e '.visibility == "private" and .project_creation_level == "noone" and
(.shared_with_groups | length) == 0' >/dev/null
glab api --hostname gitlab.com --paginate \
"groups/${GSS_GROUP_ID}/members/all?per_page=100" \
| jq -s 'add | map({id, username, access_level, expires_at})'
rm -f "${GSS_PROJECTS_FILE}" "${GSS_DESCENDANTS_FILE}"
Replace the bootstrap branch protection with one exact rule. The request body uses the documented Protected Branches API object arrays; don't replace the Owner user with a Maintainer role level. Update the existing rule atomically so the branch never passes through an unprotected interval.
set -euo pipefail
GSS_PROJECT_READ="$(mktemp)"
glab api --hostname gitlab.com "projects/${GSS_PROJECT_ID}" \
>"${GSS_PROJECT_READ}"
jq -e --argjson project "${GSS_PROJECT_ID}" --argjson group "${GSS_GROUP_ID}" '
.id == $project and .namespace.id == $group and .visibility == "private" and
.default_branch == "main" and .auto_devops_enabled == false and
(.shared_with_groups | length) == 0' "${GSS_PROJECT_READ}" >/dev/null
rm -f "${GSS_PROJECT_READ}"
GSS_VARIABLES_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"projects/${GSS_PROJECT_ID}/variables?per_page=100" \
| jq -s 'add' >"${GSS_VARIABLES_FILE}"
test "$(jq 'length' "${GSS_VARIABLES_FILE}")" -eq 0
rm -f "${GSS_VARIABLES_FILE}"
GSS_CURRENT_GROUP_ID="${GSS_GROUP_ID}"
while test "${GSS_CURRENT_GROUP_ID}" != null; do
glab api --hostname gitlab.com --paginate \
"groups/${GSS_CURRENT_GROUP_ID}/variables?per_page=100" \
| jq -s 'add | map({key, variable_type, protected, masked, hidden,
environment_scope})'
GSS_CURRENT_GROUP_ID="$(glab api --hostname gitlab.com \
"groups/${GSS_CURRENT_GROUP_ID}" | jq -r '.parent_id')"
done
unset GSS_CURRENT_GROUP_ID
GSS_TREE_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"projects/${GSS_PROJECT_ID}/repository/tree?ref=main&recursive=true&per_page=100" \
| jq -s 'add' >"${GSS_TREE_FILE}"
jq -e 'length == 1 and .[0].path == "README.md" and .[0].type == "blob"' \
"${GSS_TREE_FILE}" >/dev/null
rm -f "${GSS_TREE_FILE}"
GSS_BOOTSTRAP_RULE="$(mktemp)"
glab api --hostname gitlab.com \
"projects/${GSS_PROJECT_ID}/protected_branches/main" \
>"${GSS_BOOTSTRAP_RULE}"
GSS_PROTECTION_FILE="$(mktemp)"
jq --argjson owner "${GSS_OWNER_USER_ID}" '{
allow_force_push: false,
allowed_to_push: ([.push_access_levels[] | {id, _destroy: true}] +
[{user_id: $owner}]),
allowed_to_merge: ([.merge_access_levels[] | {id, _destroy: true}] +
[{user_id: $owner}]),
allowed_to_unprotect: ([.unprotect_access_levels[] | {id, _destroy: true}] +
[{user_id: $owner}])
}' "${GSS_BOOTSTRAP_RULE}" >"${GSS_PROTECTION_FILE}"
glab api --hostname gitlab.com --method PATCH \
"projects/${GSS_PROJECT_ID}/protected_branches/main" \
--header "Content-Type: application/json" \
--input "${GSS_PROTECTION_FILE}" >/dev/null
rm -f "${GSS_BOOTSTRAP_RULE}" "${GSS_PROTECTION_FILE}"
GSS_PROTECTION_READ="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"projects/${GSS_PROJECT_ID}/protected_branches?per_page=100" \
| jq -s 'add' >"${GSS_PROTECTION_READ}"
jq 'map({name, allow_force_push,
push: [.push_access_levels[] | {user_id, access_level}],
merge: [.merge_access_levels[] | {user_id, access_level}],
unprotect: [.unprotect_access_levels[] | {user_id, access_level}]})' \
"${GSS_PROTECTION_READ}"
jq -e --argjson owner "${GSS_OWNER_USER_ID}" '
length == 1 and .[0].name == "main" and
.[0].allow_force_push == false and
(.[] | .push_access_levels | length) == 1 and
(.[] | .push_access_levels[0].user_id) == $owner and
(.[] | .merge_access_levels | length) == 1 and
(.[] | .merge_access_levels[0].user_id) == $owner and
(.[] | .unprotect_access_levels | length) == 1 and
(.[] | .unprotect_access_levels[0].user_id) == $owner' \
"${GSS_PROTECTION_READ}" >/dev/null
rm -f "${GSS_PROTECTION_READ}"
Check for exactly one matching main rule, allow_force_push: false, and the Owner's user ID as the sole push, merge, and unprotect entry. Stop on a wildcard rule or another matching grant.
Prepare immutable local components
These pinned references target the selected Linux ARM64 Docker context. Check their resolved platform and local image ID before use. Don't assume that every sha256 in a reference is a single-platform manifest:
export GSS_RUNNER_IMAGE="gitlab/gitlab-runner@sha256:310355c898a98e0fb67d63550af9989290100abf65b5662e749790f76f5fd482"
export GSS_HELPER_IMAGE="registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper@sha256:f07e26932a43346281863ea24ae91e457bd657496557faa6ee8e0a95af3ab481"
export GSS_JOB_IMAGE="python@sha256:f77ac9e44ae96ef2c90b8053ea08c31f8be030f824196b0ae4db6d462c84e51f"
export GSS_PROXY_IMAGE="ubuntu/squid@sha256:2580be07409d64c1b5bff32d218649771a53616b06ef102bc3005e81534beccf"
export GSS_INTERNAL_NETWORK="gss-runner-internal"
export GSS_OUTBOUND_NETWORK="gss-runner-outbound"
export GSS_INTERNAL_SUBNET="172.30.248.0/28"
export GSS_PROXY_NAME="gss-runner-proxy"
export GSS_MANAGER_NAME="gss-runner-manager"
export GSS_PROXY_IP="172.30.248.2"
export GSS_SCOPE_LABEL="gss.exercise=dedicated-runner-practice"
Inventory existing containers, networks, routes, and volumes. Select a private subnet that doesn't overlap a Docker, host, VPN, or organization route. The commands below use 172.30.248.0/28 only as an example. Preserve unrelated objects on a shared Docker host.
Check that the fixed names and chosen subnet are absent before creating anything. Only Docker's exact not-found response establishes absence; a daemon or context failure leaves the result unresolved. The exercise label must also be unused because cleanup uses it to find transient build and helper containers:
set -euo pipefail
for name in "${GSS_PROXY_NAME}" "${GSS_MANAGER_NAME}"; do
GSS_ABSENCE="$(mktemp)"
if gss_docker inspect "${name}" >"${GSS_ABSENCE}" 2>&1; then
printf 'Container name already exists: %s\n' "${name}" >&2
exit 1
fi
grep -Eqi 'No such (object|container)' "${GSS_ABSENCE}"
rm -f "${GSS_ABSENCE}"
done
for name in "${GSS_INTERNAL_NETWORK}" "${GSS_OUTBOUND_NETWORK}"; do
GSS_ABSENCE="$(mktemp)"
if gss_docker network inspect "${name}" >"${GSS_ABSENCE}" 2>&1; then
printf 'Network name already exists: %s\n' "${name}" >&2
exit 1
fi
grep -Eqi 'network .+ not found|[Nn]o such network' "${GSS_ABSENCE}"
rm -f "${GSS_ABSENCE}"
done
GSS_EXISTING_SCOPED_CONTAINERS="$(gss_docker ps -aq \
--filter "label=${GSS_SCOPE_LABEL}")"
test -z "${GSS_EXISTING_SCOPED_CONTAINERS}"
unset GSS_EXISTING_SCOPED_CONTAINERS
gss_record docker_scope_absent_before_create true
gss_docker network ls --format '{{.ID}} {{.Name}} {{.Driver}}'
GSS_BASELINE_VOLUMES_FILE="${GSS_RECOVERY_DIR}/baseline-volumes.txt"
GSS_NETWORKS_FILE="$(mktemp)"
GSS_NETWORK_IDS="$(gss_docker network ls -q)"
if test -n "${GSS_NETWORK_IDS}"; then
# Word splitting is intentional for Docker's list of opaque network IDs.
# shellcheck disable=SC2086
gss_docker network inspect ${GSS_NETWORK_IDS} >"${GSS_NETWORKS_FILE}"
else
printf '[]\n' >"${GSS_NETWORKS_FILE}"
fi
python3 - "${GSS_INTERNAL_SUBNET}" "${GSS_NETWORKS_FILE}" <<'PY'
import ipaddress
import json
import sys
chosen = ipaddress.ip_network(sys.argv[1], strict=True)
networks = json.load(open(sys.argv[2], encoding="utf-8"))
existing = []
for network in networks:
for config in (network.get("IPAM", {}).get("Config") or []):
subnet = config.get("Subnet")
if subnet:
candidate = ipaddress.ip_network(subnet, strict=False)
if candidate.version == 4:
existing.append(candidate)
if any(chosen.overlaps(candidate) for candidate in existing):
raise SystemExit("chosen subnet overlaps an existing Docker network")
PY
rm -f "${GSS_NETWORKS_FILE}"
netstat -rn -f inet
Review the final route table output and stop if the chosen subnet overlaps a host, VPN, or organization route. The Docker check doesn't cover those routes.
Pull each digest deliberately and require linux/arm64 from gss_docker image inspect. With pull_policy = "never", the job fails instead of downloading unreviewed replacement bytes. GitLab documents these fields in the Docker executor and [runners.docker] configuration.
set -euo pipefail
for image in "${GSS_RUNNER_IMAGE}" "${GSS_HELPER_IMAGE}" \
"${GSS_JOB_IMAGE}" "${GSS_PROXY_IMAGE}"; do
gss_docker pull "${image}"
GSS_IMAGE_FILE="$(mktemp)"
gss_docker image inspect "${image}" >"${GSS_IMAGE_FILE}"
jq -e --arg ref "${image}" '
length == 1 and .[0].Os == "linux" and .[0].Architecture == "arm64" and
any(.[0].RepoDigests[]?; . == $ref)' "${GSS_IMAGE_FILE}" >/dev/null
jq '.[0] | {Id, RepoDigests, Os, Architecture}' "${GSS_IMAGE_FILE}"
rm -f "${GSS_IMAGE_FILE}"
done
GSS_RUNNER_LOCAL_ID="$(gss_docker image inspect "${GSS_RUNNER_IMAGE}" --format '{{.Id}}')"
GSS_HELPER_LOCAL_ID="$(gss_docker image inspect "${GSS_HELPER_IMAGE}" --format '{{.Id}}')"
GSS_JOB_LOCAL_ID="$(gss_docker image inspect "${GSS_JOB_IMAGE}" --format '{{.Id}}')"
GSS_PROXY_LOCAL_ID="$(gss_docker image inspect "${GSS_PROXY_IMAGE}" --format '{{.Id}}')"
Copy squid.conf to the trusted host and verify its reviewed hash before mounting it. It permits only unauthenticated CONNECT requests to gitlab.com:443, performs no TLS interception, and logs only method, authority, and status. It intentionally excludes every other GitLab subdomain, registry, and storage host. A checkout or artifact redirect to another authority is a new dependency to review, not a reason to broaden the list automatically.
python3 - <<'PY'
from hashlib import sha256
from pathlib import Path
import os
path = Path(os.environ["GSS_GUIDE_CHECKOUT"]) / "examples/dedicated-runner/squid.conf"
expected = "0d316d5cc81929a9d582be2cc41bd505b921a9852e12a1b04a39a0d658e38da3"
if sha256(path.read_bytes()).hexdigest() != expected:
raise SystemExit("reviewed Squid configuration differs")
PY
Create the paused group runner and protect its token
GitLab's runner creation tutorial and Runners API describe POST /user/runners. Create the runner once. If the response is lost, reconcile by fixed description and group inventory; don't replay a possibly successful creation request because GitLab returns the authentication token only once.
set -euo pipefail
umask 077
GSS_RUNNERS_BEFORE_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"groups/${GSS_GROUP_ID}/runners?type=group_type&per_page=100" \
| jq -s 'add' >"${GSS_RUNNERS_BEFORE_FILE}"
test "$(jq '[.[] | select(.description == "gss-isolated-arm64")] | length' \
"${GSS_RUNNERS_BEFORE_FILE}")" -eq 0
rm -f "${GSS_RUNNERS_BEFORE_FILE}"
gss_record runner_description_absent_before_create true
GSS_RUNNER_JSON="$(glab api --hostname gitlab.com --method POST user/runners \
--raw-field runner_type=group_type \
--raw-field "group_id=${GSS_GROUP_ID}" \
--raw-field description=gss-isolated-arm64 \
--raw-field "tag_list=${GSS_RUNNER_TAG}" \
--raw-field run_untagged=false \
--raw-field paused=true \
--raw-field access_level=ref_protected \
--raw-field maximum_timeout=1200)"
GSS_RUNNER_ID="$(jq -er '.id' <<<"${GSS_RUNNER_JSON}")"
GSS_RUNNER_TOKEN="$(jq -er '.token | select(startswith("glrt-"))' \
<<<"${GSS_RUNNER_JSON}")"
gss_record runner_id "${GSS_RUNNER_ID}"
unset GSS_RUNNER_JSON
Write the token only to a mode-0600 Runner configuration. The task container uses the reserved proxy IP, so it doesn't need job-side DNS to find the proxy. dns = ["127.0.0.1"] configures Docker's upstream; on a custom network the container normally sees Docker's embedded resolver at 127.0.0.11.
cat >"${GSS_PRIVATE_DIR}/config.toml" <<EOF_CONFIG
concurrent = 1
check_interval = 3
[[runners]]
name = "${GSS_RUNNER_TAG}"
url = "https://gitlab.com"
token = "${GSS_RUNNER_TOKEN}"
executor = "docker"
environment = [
"HTTP_PROXY=http://${GSS_PROXY_IP}:3128",
"HTTPS_PROXY=http://${GSS_PROXY_IP}:3128",
"NO_PROXY=127.0.0.1,localhost,${GSS_PROXY_IP}",
"http_proxy=http://${GSS_PROXY_IP}:3128",
"https_proxy=http://${GSS_PROXY_IP}:3128",
"no_proxy=127.0.0.1,localhost,${GSS_PROXY_IP}",
"GSS_EGRESS_PROXY=http://${GSS_PROXY_IP}:3128",
"GSS_EXPECTED_UID=65532",
"GSS_EXPECTED_GID=65532",
"GSS_FORBIDDEN_AUTHORITY=forbidden.invalid:443",
"GSS_RUNNER_TAG=${GSS_RUNNER_TAG}"
]
[runners.docker]
image = "${GSS_JOB_IMAGE}"
helper_image = "${GSS_HELPER_IMAGE}"
allowed_images = ["${GSS_JOB_IMAGE}"]
allowed_services = ["gss-no-services.invalid/**"]
pull_policy = "never"
allowed_pull_policies = ["never"]
network_mode = "${GSS_INTERNAL_NETWORK}"
dns = ["127.0.0.1"]
user = "65532:65532"
allowed_users = ["65532:65532"]
privileged = false
services_privileged = false
services_limit = 0
disable_cache = true
cap_drop = ["ALL"]
security_opt = ["no-new-privileges"]
memory = "256m"
memory_swap = "256m"
cpus = "1"
shm_size = 16777216
volumes = []
[runners.docker.container_labels]
"gss.exercise" = "dedicated-runner-practice"
EOF_CONFIG
chmod 600 "${GSS_PRIVATE_DIR}/config.toml"
unset GSS_RUNNER_TOKEN
Runner 18.10.1 has no supported Docker-executor pids_limit setting. Don't add that key or substitute an nproc ulimit and call it the same control. This procedure establishes no PID ceiling.
Start the proxy and manager
Create two networks. Only the proxy joins the outbound network. The manager is a trusted control-plane container and mounts the Docker socket; build and helper containers must never mount it.
set -euo pipefail
gss_docker network create --internal --subnet "${GSS_INTERNAL_SUBNET}" \
--label "${GSS_SCOPE_LABEL}" "${GSS_INTERNAL_NETWORK}"
gss_docker network create --label "${GSS_SCOPE_LABEL}" "${GSS_OUTBOUND_NETWORK}"
gss_docker run -d --name "${GSS_PROXY_NAME}" --label "${GSS_SCOPE_LABEL}" \
--hostname gss-runner-proxy --network "${GSS_INTERNAL_NETWORK}" \
--ip "${GSS_PROXY_IP}" --read-only --user 13:13 --cap-drop ALL \
--security-opt no-new-privileges --memory 256m --memory-swap 256m \
--pids-limit 32 --cpus 0.5 \
--tmpfs /tmp:rw,noexec,nosuid,size=8m,uid=13,gid=13,mode=0700 \
--tmpfs /var/spool/squid:rw,noexec,nosuid,size=8m,uid=13,gid=13,mode=0700 \
--tmpfs /var/log/squid:rw,noexec,nosuid,size=8m,uid=13,gid=13,mode=0700 \
--mount "type=bind,src=${GSS_GUIDE_CHECKOUT}/examples/dedicated-runner/squid.conf,dst=/etc/squid/squid.conf,readonly" \
--entrypoint /usr/sbin/squid "${GSS_PROXY_IMAGE}" \
-f /etc/squid/squid.conf -NYC
gss_docker network connect "${GSS_OUTBOUND_NETWORK}" "${GSS_PROXY_NAME}"
gss_docker run -d --name "${GSS_MANAGER_NAME}" --label "${GSS_SCOPE_LABEL}" \
--restart no \
--mount "type=bind,src=${GSS_PRIVATE_DIR},dst=/etc/gitlab-runner" \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \
"${GSS_RUNNER_IMAGE}" run --working-directory /home/gitlab-runner \
--config /etc/gitlab-runner/config.toml
GSS_PROXY_ID="$(gss_docker inspect "${GSS_PROXY_NAME}" --format '{{.Id}}')"
GSS_MANAGER_ID="$(gss_docker inspect "${GSS_MANAGER_NAME}" --format '{{.Id}}')"
GSS_INTERNAL_NETWORK_ID="$(gss_docker network inspect "${GSS_INTERNAL_NETWORK}" --format '{{.Id}}')"
GSS_OUTBOUND_NETWORK_ID="$(gss_docker network inspect "${GSS_OUTBOUND_NETWORK}" --format '{{.Id}}')"
gss_record proxy_id "${GSS_PROXY_ID}"
gss_record manager_id "${GSS_MANAGER_ID}"
gss_record internal_network_id "${GSS_INTERNAL_NETWORK_ID}"
gss_record outbound_network_id "${GSS_OUTBOUND_NETWORK_ID}"
Read the runner detail endpoint because the group's shorter runner list omits some of the fields needed here. Check the expected description, sole tag, run_untagged: false, access_level: ref_protected, and paused: true. A connected paused runner can correctly report online: true and status: paused; the paused field is the administrative gate. Read /runners/:id/managers and require the intended version, Linux platform, ARM64 architecture, and one current manager.
set -euo pipefail
GSS_RUNNER_FILE="$(mktemp)"
GSS_MANAGERS_FILE="$(mktemp)"
for attempt in $(seq 1 20); do
glab api --hostname gitlab.com "runners/${GSS_RUNNER_ID}" \
>"${GSS_RUNNER_FILE}"
glab api --hostname gitlab.com "runners/${GSS_RUNNER_ID}/managers" \
>"${GSS_MANAGERS_FILE}"
if jq -e --argjson id "${GSS_RUNNER_ID}" --arg tag "${GSS_RUNNER_TAG}" '
.id == $id and .description == "gss-isolated-arm64" and
.paused == true and .online == true and .run_untagged == false and
.access_level == "ref_protected" and .tag_list == [$tag]' \
"${GSS_RUNNER_FILE}" >/dev/null \
&& jq -e 'length == 1 and .[0].version == "18.10.1" and
.[0].platform == "linux" and .[0].architecture == "arm64" and
.[0].status == "online"' "${GSS_MANAGERS_FILE}" >/dev/null; then
break
fi
test "${attempt}" -lt 20
sleep 3
done
jq '{id, description, paused, online, status, run_untagged,
access_level, tag_list, version, platform, architecture}' \
"${GSS_RUNNER_FILE}"
jq '[.[] | {id, system_id, version, platform, architecture,
contacted_at, status}]' "${GSS_MANAGERS_FILE}"
test "$(jq 'length' "${GSS_MANAGERS_FILE}")" -eq 1
GSS_RUNNER_MANAGER_ID="$(jq -er '.[0].id' "${GSS_MANAGERS_FILE}")"
rm -f "${GSS_RUNNER_FILE}" "${GSS_MANAGERS_FILE}"
Once those reads and the protected-branch readback match the intended setup, unpause the runner:
glab api --hostname gitlab.com --method PUT "runners/${GSS_RUNNER_ID}" \
--raw-field paused=false >/dev/null
Run and inspect the single protected job
Start a host-side observer before the push. The CI example keeps the task container alive for 12 seconds after writing its report so the observer has a stable opportunity to capture both build and predefined helper settings. The observer retains selected Docker fields and environment variable names only:
set -euo pipefail
GSS_OBSERVATION_DIR="${GSS_RECOVERY_DIR}/observations"
mkdir -p "${GSS_OBSERVATION_DIR}"
GSS_OBSERVER_STOP="${GSS_OBSERVATION_DIR}/stop"
(
while test ! -e "${GSS_OBSERVER_STOP}"; do
GSS_RUNNING_IDS="${GSS_OBSERVATION_DIR}/running-ids"
gss_docker ps -q --filter "label=gss.exercise=dedicated-runner-practice" \
>"${GSS_RUNNING_IDS}"
while read -r container_id; do
test -n "${container_id}" || continue
GSS_RAW_INSPECT="${GSS_OBSERVATION_DIR}/${container_id}.raw"
GSS_FINAL_INSPECT="${GSS_OBSERVATION_DIR}/${container_id}.json"
if gss_docker inspect "${container_id}" >"${GSS_RAW_INSPECT}" 2>/dev/null; then
if test ! -e "${GSS_FINAL_INSPECT}"; then
jq '[.[0] | select(.State.Running == true) | {
Id, Name, Image,
config: {image: .Config.Image, user: .Config.User,
env_names: [.Config.Env[]? | split("=")[0]], labels: .Config.Labels},
host: {privileged: .HostConfig.Privileged,
readonly_rootfs: .HostConfig.ReadonlyRootfs,
cap_add: .HostConfig.CapAdd, cap_drop: .HostConfig.CapDrop,
security_opt: .HostConfig.SecurityOpt,
memory: .HostConfig.Memory, memory_swap: .HostConfig.MemorySwap,
nano_cpus: .HostConfig.NanoCpus, pids_limit: .HostConfig.PidsLimit,
dns: .HostConfig.Dns, network_mode: .HostConfig.NetworkMode},
mounts: [.Mounts[]? | {Type, Name, Destination, RW}],
networks: (.NetworkSettings.Networks | keys),
state: {status: .State.Status, running: .State.Running,
exit_code: .State.ExitCode, oom_killed: .State.OOMKilled}
}]' "${GSS_RAW_INSPECT}" >"${GSS_FINAL_INSPECT}.tmp"
if test "$(jq 'length' "${GSS_FINAL_INSPECT}.tmp")" -eq 1; then
mv "${GSS_FINAL_INSPECT}.tmp" "${GSS_FINAL_INSPECT}"
else
rm -f "${GSS_FINAL_INSPECT}.tmp"
fi
fi
fi
rm -f "${GSS_RAW_INSPECT}"
done <"${GSS_RUNNING_IDS}"
sleep 0.2
done
) &
GSS_OBSERVER_PID="$!"
gss_stop_observer() {
touch "${GSS_OBSERVER_STOP}"
wait "${GSS_OBSERVER_PID}" || true
}
trap gss_stop_observer EXIT
From a clean clone of the new project, replace <unique-protected-runner-tag> in .gitlab-ci.yml with GSS_RUNNER_TAG, then add the CI file and probe.py. Commit and push them to the protected main as the Owner. Confirm the local source hashes before the push.
set -euo pipefail
git clone "${GSS_PROJECT_HTTP_URL}" runner-boundary
cd runner-boundary
mkdir -p examples/dedicated-runner
python3 - <<'PY'
from hashlib import sha256
from pathlib import Path
expected = {
"examples/dedicated-runner/probe.py": "d78c80da4b23d4712903d3007ad289e47c4f0f0a5d85ab6b72376e04f2078dce",
"examples/dedicated-runner/.gitlab-ci.yml": "5e8b28b454187b1e127fd92d9d170f835d4434c350cc3e9b31a6f3c050638b81",
}
root = Path(__import__("os").environ["GSS_GUIDE_CHECKOUT"])
for relative, wanted in expected.items():
actual = sha256((root / relative).read_bytes()).hexdigest()
if actual != wanted:
raise SystemExit(f"reviewed source differs: {relative}")
PY
cp "${GSS_GUIDE_CHECKOUT}/examples/dedicated-runner/probe.py" \
examples/dedicated-runner/probe.py
sed "s/<unique-protected-runner-tag>/${GSS_RUNNER_TAG}/" \
"${GSS_GUIDE_CHECKOUT}/examples/dedicated-runner/.gitlab-ci.yml" \
>.gitlab-ci.yml
git add .gitlab-ci.yml examples/dedicated-runner/probe.py
git diff --cached
git commit -m "test: exercise protected dedicated runner"
GSS_PROBE_SHA="$(git rev-parse HEAD)"
gss_record probe_sha "${GSS_PROBE_SHA}"
git push origin HEAD:main
Poll only for the exact push pipeline. Stop on a second matching pipeline, timeout, failed pipeline, wrong actor or unexpected job. This block also binds the job to the runner and manager selected before unpause and downloads the single report artifact:
set -euo pipefail
GSS_PIPELINE_FILE="$(mktemp)"
GSS_PIPELINE_ID=""
for attempt in $(seq 1 60); do
GSS_PIPELINES_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"projects/${GSS_PROJECT_ID}/pipelines?sha=${GSS_PROBE_SHA}&ref=main&source=push&per_page=100" \
| jq -s 'add' >"${GSS_PIPELINES_FILE}"
test "$(jq 'length' "${GSS_PIPELINES_FILE}")" -le 1
if test "$(jq 'length' "${GSS_PIPELINES_FILE}")" -eq 1; then
GSS_PIPELINE_ID="$(jq -er '.[0].id' "${GSS_PIPELINES_FILE}")"
glab api --hostname gitlab.com \
"projects/${GSS_PROJECT_ID}/pipelines/${GSS_PIPELINE_ID}" \
>"${GSS_PIPELINE_FILE}"
GSS_PIPELINE_STATUS="$(jq -er '.status' "${GSS_PIPELINE_FILE}")"
case "${GSS_PIPELINE_STATUS}" in
success) break ;;
failed|canceled|skipped|manual)
printf 'Pipeline stopped with status %s.\n' "${GSS_PIPELINE_STATUS}" >&2
exit 1
;;
esac
fi
rm -f "${GSS_PIPELINES_FILE}"
test "${attempt}" -lt 60
sleep 3
done
rm -f "${GSS_PIPELINES_FILE}"
test -n "${GSS_PIPELINE_ID}"
gss_record pipeline_id "${GSS_PIPELINE_ID}"
jq -e --arg sha "${GSS_PROBE_SHA}" --argjson owner "${GSS_OWNER_USER_ID}" '
.sha == $sha and .ref == "main" and .source == "push" and
.status == "success" and .user.id == $owner' \
"${GSS_PIPELINE_FILE}" >/dev/null
GSS_JOBS_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"projects/${GSS_PROJECT_ID}/pipelines/${GSS_PIPELINE_ID}/jobs?include_retried=true&per_page=100" \
| jq -s 'add' >"${GSS_JOBS_FILE}"
test "$(jq 'length' "${GSS_JOBS_FILE}")" -eq 1
GSS_JOB_ID="$(jq -er --arg sha "${GSS_PROBE_SHA}" \
--argjson pipeline "${GSS_PIPELINE_ID}" --argjson runner "${GSS_RUNNER_ID}" \
--argjson manager "${GSS_RUNNER_MANAGER_ID}" '
.[0] | select(.name == "dedicated-runner-boundary" and
.status == "success" and .commit.id == $sha and
.pipeline.id == $pipeline and
.runner.id == $runner and .runner_manager.id == $manager) | .id' \
"${GSS_JOBS_FILE}")"
gss_record job_id "${GSS_JOB_ID}"
GSS_REPORT_FILE="$(mktemp)"
glab api --hostname gitlab.com \
"projects/${GSS_PROJECT_ID}/jobs/${GSS_JOB_ID}/artifacts/dedicated-runner-report.json" \
>"${GSS_REPORT_FILE}"
jq -e --arg sha "${GSS_PROBE_SHA}" --arg pipeline "${GSS_PIPELINE_ID}" \
--arg job "${GSS_JOB_ID}" --arg tag "${GSS_RUNNER_TAG}" '
.schema == "gss-dedicated-runner-probe-v1" and .outcome == "passed" and
.checkout.ci_commit_sha == $sha and .checkout.head_sha == $sha and
.checkout.pipeline_source == "push" and .checkout.ref == "main" and
.checkout.source_sha256 == "d78c80da4b23d4712903d3007ad289e47c4f0f0a5d85ab6b72376e04f2078dce" and
.gitlab.pipeline_id == $pipeline and .gitlab.job_id == $job and
.runner_tag == $tag and .process.uid == 65532 and .process.gid == 65532 and
.process.no_new_privs == true and .filesystem.root_write_denied == true and
.network.loopback == "loopback-ok" and
.network.direct_off_host_errno == 101 and
.network.dns.nameservers == ["127.0.0.11"] and
.network.dns.public_lookup_errno == -3 and
.network.forbidden_connect_status == 403' "${GSS_REPORT_FILE}" >/dev/null
While the job runs, the trusted host operator records Docker events and inspects every matching running build and predefined helper container. After the exact job is terminal, stop and join the observer before reading its files:
touch "${GSS_OBSERVER_STOP}"
wait "${GSS_OBSERVER_PID}"
trap - EXIT
jq . "${GSS_OBSERVATION_DIR}"/*.json
jq -e -s --arg build "${GSS_JOB_LOCAL_ID}" \
--arg helper "${GSS_HELPER_LOCAL_ID}" '
[ .[][] | select(.Name | endswith("-build")) ] as $builds |
[ .[][] | select(.Name | endswith("-predefined")) ] as $helpers |
($builds | length) >= 1 and ($helpers | length) >= 1 and
all($builds[];
.Image == $build and .config.user == "65532:65532" and
.host.privileged == false and .host.cap_drop == ["ALL"] and
any(.host.security_opt[]?; contains("no-new-privileges")) and
.host.memory == 268435456 and .host.memory_swap == 268435456 and
.host.nano_cpus == 1000000000 and
.host.network_mode == env.GSS_INTERNAL_NETWORK and
.host.dns == ["127.0.0.1"] and
all(.mounts[]?; .Destination != "/var/run/docker.sock")) and
all($helpers[];
.Image == $helper and .host.privileged == false and
.host.cap_drop == ["ALL"] and
any(.host.security_opt[]?; contains("no-new-privileges")) and
.host.network_mode == env.GSS_INTERNAL_NETWORK and
.host.dns == ["127.0.0.1"] and
all(.mounts[]?; .Destination != "/var/run/docker.sock"))' \
"${GSS_OBSERVATION_DIR}"/*.json >/dev/null
GSS_PROXY_LOG="$(mktemp)"
gss_docker exec "${GSS_PROXY_NAME}" \
cat /var/log/squid/access.log >"${GSS_PROXY_LOG}"
test -s "${GSS_PROXY_LOG}"
if grep -Ev '^CONNECT [^/?# ]+:443 (200|403)$' "${GSS_PROXY_LOG}"; then
printf 'Unexpected proxy log entry.\n' >&2
exit 1
fi
grep -Fx 'CONNECT gitlab.com:443 200' "${GSS_PROXY_LOG}" >/dev/null
grep -Fx 'CONNECT forbidden.invalid:443 403' "${GSS_PROXY_LOG}" >/dev/null
Check for at least one complete running snapshot for both the -build and -predefined containers. Docker's container Image field is a local image ID, not necessarily the registry digest written in name@sha256:.... Compare it with GSS_JOB_LOCAL_ID or GSS_HELPER_LOCAL_ID, which came from gss_docker image inspect on the reviewed references. Record only environment variable names, never values.
Together, the records should show all of the following:
- The pipeline is a successful
pushpipeline on protectedmainatGSS_PROBE_SHA, with exactly onededicated-runner-boundaryjob. - The job names
GSS_RUNNER_IDand the manager ID read before unpause. - The artifact's checkout SHA, pipeline/job IDs, source hash, ref, and runner tag match GitLab's records.
- The build container uses UID/GID
65532, zero effective capabilities,no-new-privileges, the exact internal network, the expected Docker embedded resolver, 256 MiB memory and swap, one CPU, and no Docker socket. - The helper uses the pinned helper digest, exact internal network, restricted DNS, dropped capabilities,
no-new-privileges, and no Docker socket. Its user is a separate observation; this configuration doesn't force the build user's identity onto the helper. - The report shows the bounded workspace write, denied root-level write, loopback success, direct off-host
ENETUNREACH, failed public DNS lookup, forbidden CONNECT403, and no secret values. - The proxy log contains
CONNECT gitlab.com:443 200for GitLab traffic andCONNECT forbidden.invalid:443 403, with no unexpected authority.
Download dedicated-runner-report.json through the Owner session and compare it with the GitLab pipeline/job API responses. A green pipeline alone isn't enough. The tag selects eligible runners; the job's runner.id and runner_manager.id show which manager actually ran it.
Diagnose a stopped setup
- Runner is
online: true,status: paused. The manager is connected and the administrative pause is working. Complete the readbacks before unpausing. - The group runner list omits tags or manager details. Use the exact runner detail and
/runners/:id/managersendpoints. Don't weaken the expected tag or protection because the summary response is shorter. - Checkout or artifact upload reaches another host. Keep the job stopped. Record the authority and decide whether that dependency should be admitted; don't add a wildcard GitLab domain.
- The task reports a DNS timeout or an unexpected direct-connect result. Treat it as an unresolved network result. The
127.0.0.11resolver address by itself isn't proof that Docker didn't forward a query. - Docker reports
PidsLimit: null. That's expected for this configuration: Runner 18.10.1 has nopids_limitfield. This configuration has no PID ceiling. - An API mutation times out. Stop and reconcile the exact group, project, runner, branch, or pipeline identity before retrying. A timeout isn't proof that a write failed.
Recover from a fresh shell
Use this entry point if setup stopped, the terminal closed, or a mutation's response was lost. It authenticates the same Owner, reloads durable local scope, and rediscovers only resources with the fixed identities chosen above. It doesn't create or retry anything.
set -euo pipefail
export GSS_RECOVERY_DIR="<same-absolute-private-path>/gss-dedicated-runner-recovery"
export GSS_RECOVERY_FILE="${GSS_RECOVERY_DIR}/recovery.json"
jq -e 'type == "object" and .parent_group_id and .parent_full_path and
.owner_user_id and .subgroup_path and .project_path and .runner_tag and
.docker_context and .private_dir and
.group_path_absent_before_create == true' "${GSS_RECOVERY_FILE}" >/dev/null
export GSS_PARENT_GROUP_ID="$(jq -er '.parent_group_id' "${GSS_RECOVERY_FILE}")"
export GSS_OWNER_USER_ID="$(jq -er '.owner_user_id' "${GSS_RECOVERY_FILE}")"
export GSS_PARENT_FULL_PATH="$(jq -er '.parent_full_path' "${GSS_RECOVERY_FILE}")"
export GSS_SUBGROUP_PATH="$(jq -er '.subgroup_path' "${GSS_RECOVERY_FILE}")"
export GSS_PROJECT_PATH="$(jq -er '.project_path' "${GSS_RECOVERY_FILE}")"
export GSS_RUNNER_TAG="$(jq -er '.runner_tag' "${GSS_RECOVERY_FILE}")"
export GSS_REVIEW_BY="$(jq -er '.review_by' "${GSS_RECOVERY_FILE}")"
export GSS_DOCKER_CONTEXT="$(jq -er '.docker_context' "${GSS_RECOVERY_FILE}")"
export GSS_PRIVATE_DIR="$(jq -er '.private_dir' "${GSS_RECOVERY_FILE}")"
export GSS_PROBE_SHA="$(jq -r '.probe_sha // empty' "${GSS_RECOVERY_FILE}")"
export GSS_JOB_ID="$(jq -r '.job_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_RECORDED_GROUP_ID="$(jq -r '.group_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_RECORDED_PROJECT_ID="$(jq -r '.project_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_RECORDED_RUNNER_ID="$(jq -r '.runner_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_RECORDED_PROXY_ID="$(jq -r '.proxy_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_RECORDED_MANAGER_ID="$(jq -r '.manager_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_RECORDED_INTERNAL_NETWORK_ID="$(jq -r '.internal_network_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_RECORDED_OUTBOUND_NETWORK_ID="$(jq -r '.outbound_network_id // empty' "${GSS_RECOVERY_FILE}")"
export GSS_DOCKER_SCOPE_ABSENT="$(jq -r '.docker_scope_absent_before_create // empty' "${GSS_RECOVERY_FILE}")"
export GSS_BASELINE_VOLUMES_FILE="${GSS_RECOVERY_DIR}/baseline-volumes.txt"
export GSS_OBSERVATION_DIR="${GSS_RECOVERY_DIR}/observations"
export GSS_INTERNAL_NETWORK="gss-runner-internal"
export GSS_OUTBOUND_NETWORK="gss-runner-outbound"
export GSS_PROXY_NAME="gss-runner-proxy"
export GSS_MANAGER_NAME="gss-runner-manager"
export GSS_SCOPE_LABEL="gss.exercise=dedicated-runner-practice"
gss_docker() { docker --context "${GSS_DOCKER_CONTEXT}" "$@"; }
test "$(glab api --hostname gitlab.com user | jq -er '.id')" \
= "${GSS_OWNER_USER_ID}"
glab api --hostname gitlab.com "groups/${GSS_PARENT_GROUP_ID}" \
| jq -e --argjson id "${GSS_PARENT_GROUP_ID}" \
--arg path "${GSS_PARENT_FULL_PATH}" \
'.id == $id and .full_path == $path' >/dev/null
docker context inspect "${GSS_DOCKER_CONTEXT}" >/dev/null
gss_docker info >/dev/null
gss_match_or_absent() {
recorded="$1"; discovered="$2"; endpoint="$3"; missing_message="$4"
test -n "${recorded}" || return 0
if test -n "${discovered}"; then
test "${discovered}" = "${recorded}"
else
absence="$(mktemp)"
if glab api --hostname gitlab.com "${endpoint}/${recorded}" \
>"${absence}" 2>&1; then
printf 'Recorded resource still exists outside the expected scope: %s\n' \
"${endpoint}/${recorded}" >&2
return 1
fi
grep -F "${missing_message}" "${absence}" >/dev/null
rm -f "${absence}"
fi
}
GSS_EXPECTED_GROUP_PATH="${GSS_PARENT_FULL_PATH}/${GSS_SUBGROUP_PATH}"
GSS_GROUPS_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"groups/${GSS_PARENT_GROUP_ID}/subgroups?per_page=100" \
| jq -s 'add' >"${GSS_GROUPS_FILE}"
test "$(jq --arg path "${GSS_EXPECTED_GROUP_PATH}" \
'[.[] | select(.full_path == $path)] | length' "${GSS_GROUPS_FILE}")" -le 1
GSS_GROUP_ID="$(jq -r --arg path "${GSS_EXPECTED_GROUP_PATH}" \
--argjson parent "${GSS_PARENT_GROUP_ID}" '
[.[] | select(.full_path == $path and .parent_id == $parent and
.visibility == "private")] | .[0].id // empty' "${GSS_GROUPS_FILE}")"
rm -f "${GSS_GROUPS_FILE}"
gss_match_or_absent "${GSS_RECORDED_GROUP_ID}" "${GSS_GROUP_ID}" \
groups '404 Group Not Found'
GSS_PROJECT_ID=""
GSS_RUNNER_ID=""
if test -n "${GSS_GROUP_ID}"; then
GSS_PROJECTS_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"groups/${GSS_GROUP_ID}/projects?include_subgroups=false&with_shared=false&per_page=100" \
| jq -s 'add' >"${GSS_PROJECTS_FILE}"
test "$(jq --arg path "${GSS_PROJECT_PATH}" \
'[.[] | select(.path == $path)] | length' "${GSS_PROJECTS_FILE}")" -le 1
GSS_PROJECT_ID="$(jq -r --arg path "${GSS_PROJECT_PATH}" \
--argjson group "${GSS_GROUP_ID}" \
'[.[] | select(.path == $path and .visibility == "private" and
.namespace.id == $group)] | .[0].id // empty' \
"${GSS_PROJECTS_FILE}")"
rm -f "${GSS_PROJECTS_FILE}"
GSS_RUNNERS_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"groups/${GSS_GROUP_ID}/runners?type=group_type&per_page=100" \
| jq -s 'add' >"${GSS_RUNNERS_FILE}"
test "$(jq '[.[] | select(.description == "gss-isolated-arm64")] | length' \
"${GSS_RUNNERS_FILE}")" -le 1
GSS_RUNNER_ID="$(jq -r '
[.[] | select(.description == "gss-isolated-arm64")] |
.[0].id // empty' "${GSS_RUNNERS_FILE}")"
rm -f "${GSS_RUNNERS_FILE}"
if test -n "${GSS_RUNNER_ID}"; then
if test -z "${GSS_RECORDED_RUNNER_ID}"; then
test "$(jq -r '.runner_description_absent_before_create // empty' \
"${GSS_RECOVERY_FILE}")" = true
fi
GSS_RUNNER_FILE="$(mktemp)"
glab api --hostname gitlab.com "runners/${GSS_RUNNER_ID}" \
>"${GSS_RUNNER_FILE}"
jq -e --argjson id "${GSS_RUNNER_ID}" --arg tag "${GSS_RUNNER_TAG}" '
.id == $id and .description == "gss-isolated-arm64" and
.run_untagged == false and .access_level == "ref_protected" and
.tag_list == [$tag]' "${GSS_RUNNER_FILE}" >/dev/null
rm -f "${GSS_RUNNER_FILE}"
fi
fi
gss_match_or_absent "${GSS_RECORDED_PROJECT_ID}" "${GSS_PROJECT_ID}" \
projects '404 Project Not Found'
gss_match_or_absent "${GSS_RECORDED_RUNNER_ID}" "${GSS_RUNNER_ID}" \
runners '404 Runner Not Found'
for pair in "${GSS_MANAGER_NAME}:${GSS_RECORDED_MANAGER_ID}" \
"${GSS_PROXY_NAME}:${GSS_RECORDED_PROXY_ID}"; do
name="${pair%%:*}"
recorded_id="${pair#*:}"
GSS_INSPECT_FILE="$(mktemp)"
if gss_docker inspect "${name}" >"${GSS_INSPECT_FILE}" 2>&1; then
if test -z "${recorded_id}"; then
test "${GSS_DOCKER_SCOPE_ABSENT}" = true
fi
jq -e 'length == 1 and
.[0].Config.Labels["gss.exercise"] == "dedicated-runner-practice"' \
"${GSS_INSPECT_FILE}" >/dev/null
if test -n "${recorded_id}"; then
test "$(jq -er '.[0].Id' "${GSS_INSPECT_FILE}")" = "${recorded_id}"
fi
else
grep -Eqi 'No such (object|container)' "${GSS_INSPECT_FILE}"
fi
rm -f "${GSS_INSPECT_FILE}"
done
for pair in "${GSS_INTERNAL_NETWORK}:${GSS_RECORDED_INTERNAL_NETWORK_ID}" \
"${GSS_OUTBOUND_NETWORK}:${GSS_RECORDED_OUTBOUND_NETWORK_ID}"; do
name="${pair%%:*}"
recorded_id="${pair#*:}"
GSS_INSPECT_FILE="$(mktemp)"
if gss_docker network inspect "${name}" \
>"${GSS_INSPECT_FILE}" 2>&1; then
if test -z "${recorded_id}"; then
test "${GSS_DOCKER_SCOPE_ABSENT}" = true
fi
jq -e 'length == 1 and
.[0].Labels["gss.exercise"] == "dedicated-runner-practice"' \
"${GSS_INSPECT_FILE}" >/dev/null
if test -n "${recorded_id}"; then
test "$(jq -er '.[0].Id' "${GSS_INSPECT_FILE}")" = "${recorded_id}"
fi
else
grep -Eqi 'network .+ not found|[Nn]o such network' "${GSS_INSPECT_FILE}"
fi
rm -f "${GSS_INSPECT_FILE}"
done
If a fixed identity returns more than one candidate, or a resource exists but fails an ownership check, stop. Preserve it as unknown and investigate before continuing. If the group or project creation response was lost, the exact path and parent readback above determine whether the write succeeded. If runner creation is ambiguous, the group inventory must yield at most one runner with the exact description, and its detail readback must match the sole tag and protected settings. A discovered runner has no recoverable authentication token; pause and remove it, then start a new setup only after cleanup is complete.
Clean up only these resources
Pause the exact runner first, then wait for its exact job to reach a terminal state. GitLab removes transient build/helper containers and may remove its managed cache volume automatically. Inventory before deletion; don't treat an already absent transient object as one your cleanup removed.
set -euo pipefail
GSS_RECORDED_MANAGER_ID="$(jq -r '.manager_id // empty' "${GSS_RECOVERY_FILE}")"
GSS_RECORDED_PROXY_ID="$(jq -r '.proxy_id // empty' "${GSS_RECOVERY_FILE}")"
GSS_RECORDED_INTERNAL_NETWORK_ID="$(jq -r '.internal_network_id // empty' "${GSS_RECOVERY_FILE}")"
GSS_RECORDED_OUTBOUND_NETWORK_ID="$(jq -r '.outbound_network_id // empty' "${GSS_RECOVERY_FILE}")"
GSS_DOCKER_SCOPE_ABSENT="$(jq -r '.docker_scope_absent_before_create // empty' "${GSS_RECOVERY_FILE}")"
test -f "${GSS_BASELINE_VOLUMES_FILE}"
test ! -L "${GSS_BASELINE_VOLUMES_FILE}"
GSS_OWNED_VOLUMES_FILE="${GSS_RECOVERY_DIR}/owned-volumes.txt"
GSS_MANAGER_VOLUMES_FILE="${GSS_RECOVERY_DIR}/manager-volumes.txt"
: >"${GSS_OWNED_VOLUMES_FILE}"
: >"${GSS_MANAGER_VOLUMES_FILE}"
if test -n "${GSS_GROUP_ID:-}"; then
glab api --hostname gitlab.com --method PUT "groups/${GSS_GROUP_ID}" \
--raw-field project_creation_level=noone >/dev/null
test "$(glab api --hostname gitlab.com "groups/${GSS_GROUP_ID}" \
| jq -er '.project_creation_level')" = noone
fi
if test -n "${GSS_RUNNER_ID:-}"; then
GSS_RUNNER_FILE="$(mktemp)"
glab api --hostname gitlab.com "runners/${GSS_RUNNER_ID}" \
>"${GSS_RUNNER_FILE}"
jq -e --argjson id "${GSS_RUNNER_ID}" --arg tag "${GSS_RUNNER_TAG}" '
.id == $id and .description == "gss-isolated-arm64" and
.tag_list == [$tag]' \
"${GSS_RUNNER_FILE}" >/dev/null
glab api --hostname gitlab.com --method PUT "runners/${GSS_RUNNER_ID}" \
--raw-field paused=true >/dev/null
test "$(glab api --hostname gitlab.com "runners/${GSS_RUNNER_ID}" \
| jq -er '.paused')" = true
rm -f "${GSS_RUNNER_FILE}"
fi
if test -n "${GSS_PROJECT_ID:-}" && test -n "${GSS_PROBE_SHA:-}" \
&& test -z "${GSS_JOB_ID:-}"; then
GSS_RECOVERY_PIPELINES_FILE="$(mktemp)"
for attempt in $(seq 1 20); do
glab api --hostname gitlab.com --paginate \
"projects/${GSS_PROJECT_ID}/pipelines?sha=${GSS_PROBE_SHA}&ref=main&source=push&per_page=100" \
| jq -s 'add' >"${GSS_RECOVERY_PIPELINES_FILE}"
test "$(jq 'length' "${GSS_RECOVERY_PIPELINES_FILE}")" -le 1
test "$(jq 'length' "${GSS_RECOVERY_PIPELINES_FILE}")" -eq 0 || break
test "${attempt}" -lt 20
sleep 3
done
GSS_RECOVERY_PIPELINE_ID="$(jq -r '.[0].id // empty' \
"${GSS_RECOVERY_PIPELINES_FILE}")"
rm -f "${GSS_RECOVERY_PIPELINES_FILE}"
if test -n "${GSS_RECOVERY_PIPELINE_ID}"; then
glab api --hostname gitlab.com \
"projects/${GSS_PROJECT_ID}/pipelines/${GSS_RECOVERY_PIPELINE_ID}" \
| jq -e --arg sha "${GSS_PROBE_SHA}" \
--argjson owner "${GSS_OWNER_USER_ID}" '
.sha == $sha and .ref == "main" and .source == "push" and
.user.id == $owner' >/dev/null
GSS_RECOVERY_JOBS_FILE="$(mktemp)"
glab api --hostname gitlab.com --paginate \
"projects/${GSS_PROJECT_ID}/pipelines/${GSS_RECOVERY_PIPELINE_ID}/jobs?include_retried=true&per_page=100" \
| jq -s 'add' >"${GSS_RECOVERY_JOBS_FILE}"
test "$(jq --arg sha "${GSS_PROBE_SHA}" '
[.[] | select(.name == "dedicated-runner-boundary" and
.commit.id == $sha)] | length' "${GSS_RECOVERY_JOBS_FILE}")" -le 1
GSS_JOB_ID="$(jq -r --arg sha "${GSS_PROBE_SHA}" '
[.[] | select(.name == "dedicated-runner-boundary" and
.commit.id == $sha)] | .[0].id // empty' \
"${GSS_RECOVERY_JOBS_FILE}")"
rm -f "${GSS_RECOVERY_JOBS_FILE}"
fi
fi
GSS_ACTIVE_JOBS_FILE="$(mktemp)"
printf '[]\n' >"${GSS_ACTIVE_JOBS_FILE}"
if test -n "${GSS_PROJECT_ID:-}" && test -n "${GSS_RUNNER_ID:-}" \
&& test -n "${GSS_JOB_ID:-}"; then
GSS_RECORDED_JOB_FILE="$(mktemp)"
glab api --hostname gitlab.com \
"projects/${GSS_PROJECT_ID}/jobs/${GSS_JOB_ID}" \
>"${GSS_RECORDED_JOB_FILE}"
jq -e --argjson job "${GSS_JOB_ID}" --arg sha "${GSS_PROBE_SHA}" \
--argjson runner "${GSS_RUNNER_ID}" '
.id == $job and .name == "dedicated-runner-boundary" and
.commit.id == $sha and (.runner == null or .runner.id == $runner)' \
"${GSS_RECORDED_JOB_FILE}" >/dev/null
jq -n --argjson job "${GSS_JOB_ID}" '[$job]' >"${GSS_ACTIVE_JOBS_FILE}"
rm -f "${GSS_RECORDED_JOB_FILE}"
while read -r job_id; do
status="$(glab api --hostname gitlab.com \
"projects/${GSS_PROJECT_ID}/jobs/${job_id}" | jq -er '.status')"
case "${status}" in
success|failed|canceled|skipped|manual) ;;
*)
glab api --hostname gitlab.com --method POST \
"projects/${GSS_PROJECT_ID}/jobs/${job_id}/cancel" >/dev/null
;;
esac
done < <(jq -r '.[]' "${GSS_ACTIVE_JOBS_FILE}")
while read -r job_id; do
for attempt in $(seq 1 40); do
status="$(glab api --hostname gitlab.com \
"projects/${GSS_PROJECT_ID}/jobs/${job_id}" | jq -er '.status')"
case "${status}" in
success|failed|canceled|skipped|manual) break ;;
esac
test "${attempt}" -lt 40
sleep 3
done
done < <(jq -r '.[]' "${GSS_ACTIVE_JOBS_FILE}")
fi
rm -f "${GSS_ACTIVE_JOBS_FILE}"
if test -d "${GSS_OBSERVATION_DIR}"; then
touch "${GSS_OBSERVATION_DIR}/stop"
for observation in "${GSS_OBSERVATION_DIR}"/*.json; do
test -e "${observation}" || continue
jq -e 'type == "array" and all(.[];
.config.labels["gss.exercise"] == "dedicated-runner-practice")' \
"${observation}" >/dev/null
jq -r '.[] | .mounts[]? | select(.Type == "volume") | .Name' \
"${observation}" >>"${GSS_OWNED_VOLUMES_FILE}"
jq -r --arg manager "${GSS_RECORDED_MANAGER_ID}" '
.[] | select(.Id == $manager or .Name == "/gss-runner-manager") |
.mounts[]? |
select(.Type == "volume") | .Name' \
"${observation}" >>"${GSS_MANAGER_VOLUMES_FILE}"
done
fi
GSS_SCOPED_CONTAINERS_FILE="$(mktemp)"
gss_docker ps -aq --no-trunc --filter "label=${GSS_SCOPE_LABEL}" \
>"${GSS_SCOPED_CONTAINERS_FILE}"
while read -r container_id; do
test -n "${container_id}" || continue
GSS_INSPECT_FILE="$(mktemp)"
gss_docker inspect "${container_id}" >"${GSS_INSPECT_FILE}"
jq -e 'length == 1 and
.[0].Config.Labels["gss.exercise"] == "dedicated-runner-practice"' \
"${GSS_INSPECT_FILE}" >/dev/null
GSS_CONTAINER_NAME="$(jq -er '.[0].Name | ltrimstr("/")' \
"${GSS_INSPECT_FILE}")"
case "${GSS_CONTAINER_NAME}" in
"${GSS_MANAGER_NAME}")
if test -n "${GSS_RECORDED_MANAGER_ID}"; then
test "${container_id}" = "${GSS_RECORDED_MANAGER_ID}"
else
test "${GSS_DOCKER_SCOPE_ABSENT}" = true
fi
;;
"${GSS_PROXY_NAME}")
if test -n "${GSS_RECORDED_PROXY_ID}"; then
test "${container_id}" = "${GSS_RECORDED_PROXY_ID}"
else
test "${GSS_DOCKER_SCOPE_ABSENT}" = true
fi
;;
esac
jq -r '.[0].Mounts[]? | select(.Type == "volume") | .Name' \
"${GSS_INSPECT_FILE}" >>"${GSS_OWNED_VOLUMES_FILE}"
if test "${GSS_CONTAINER_NAME}" = "${GSS_MANAGER_NAME}"; then
jq -r '.[0].Mounts[]? | select(.Type == "volume") | .Name' \
"${GSS_INSPECT_FILE}" >>"${GSS_MANAGER_VOLUMES_FILE}"
fi
rm -f "${GSS_INSPECT_FILE}"
done <"${GSS_SCOPED_CONTAINERS_FILE}"
sort -u -o "${GSS_OWNED_VOLUMES_FILE}" "${GSS_OWNED_VOLUMES_FILE}"
sort -u -o "${GSS_MANAGER_VOLUMES_FILE}" "${GSS_MANAGER_VOLUMES_FILE}"
while read -r volume; do
test -n "${volume}" || continue
if grep -Fx "${volume}" "${GSS_BASELINE_VOLUMES_FILE}" >/dev/null; then
printf 'Captured volume existed before this exercise: %s\n' "${volume}" >&2
exit 1
fi
done <"${GSS_OWNED_VOLUMES_FILE}"
while read -r container_id; do
test -n "${container_id}" || continue
gss_docker rm -f "${container_id}" >/dev/null
done <"${GSS_SCOPED_CONTAINERS_FILE}"
rm -f "${GSS_SCOPED_CONTAINERS_FILE}"
while read -r volume; do
test -n "${volume}" || continue
GSS_VOLUME_FILE="$(mktemp)"
if gss_docker volume inspect "${volume}" >"${GSS_VOLUME_FILE}" 2>&1; then
if ! grep -Fx "${volume}" "${GSS_MANAGER_VOLUMES_FILE}" >/dev/null; then
jq -e 'length == 1 and
.[0].Labels["com.gitlab.gitlab-runner.managed"] == "true"' \
"${GSS_VOLUME_FILE}" >/dev/null
fi
gss_docker volume rm "${volume}" >/dev/null
else
grep -qi 'no such volume' "${GSS_VOLUME_FILE}"
fi
rm -f "${GSS_VOLUME_FILE}"
done <"${GSS_OWNED_VOLUMES_FILE}"
for pair in "${GSS_INTERNAL_NETWORK}:${GSS_RECORDED_INTERNAL_NETWORK_ID}" \
"${GSS_OUTBOUND_NETWORK}:${GSS_RECORDED_OUTBOUND_NETWORK_ID}"; do
name="${pair%%:*}"
expected_id="${pair#*:}"
GSS_INSPECT_FILE="$(mktemp)"
if gss_docker network inspect "${name}" \
>"${GSS_INSPECT_FILE}" 2>&1; then
jq -e 'length == 1 and
.[0].Labels["gss.exercise"] == "dedicated-runner-practice"' \
"${GSS_INSPECT_FILE}" >/dev/null
if test -n "${expected_id}"; then
test "$(jq -er '.[0].Id' "${GSS_INSPECT_FILE}")" = "${expected_id}"
else
test "${GSS_DOCKER_SCOPE_ABSENT}" = true
fi
GSS_NETWORK_ID="$(jq -er '.[0].Id' "${GSS_INSPECT_FILE}")"
gss_docker network rm "${GSS_NETWORK_ID}" >/dev/null
else
grep -Eqi 'network .+ not found|[Nn]o such network' "${GSS_INSPECT_FILE}"
fi
rm -f "${GSS_INSPECT_FILE}"
done
if test -n "${GSS_RUNNER_ID:-}"; then
glab api --hostname gitlab.com --method DELETE \
"runners/${GSS_RUNNER_ID}" >/dev/null
GSS_RUNNER_ABSENCE="$(mktemp)"
if glab api --hostname gitlab.com "runners/${GSS_RUNNER_ID}" \
>"${GSS_RUNNER_ABSENCE}" 2>&1; then
printf 'Runner still exists; stop cleanup.\n' >&2
exit 1
fi
grep -F '404 Runner Not Found' "${GSS_RUNNER_ABSENCE}" >/dev/null
rm -f "${GSS_RUNNER_ABSENCE}"
fi
test "${GSS_PRIVATE_DIR}" = "${GSS_RECOVERY_DIR}/runner-config"
test ! -L "${GSS_PRIVATE_DIR}"
if test -d "${GSS_PRIVATE_DIR}"; then
GSS_PRIVATE_ENTRIES="$(find "${GSS_PRIVATE_DIR}" -mindepth 1 -maxdepth 1 \
-exec basename {} \; | sort)"
while read -r entry; do
test -n "${entry}" || continue
case "${entry}" in
config.toml|.runner_system_id) ;;
*)
printf 'Unexpected file in private Runner directory: %s\n' \
"${entry}" >&2
exit 1
;;
esac
done <<<"${GSS_PRIVATE_ENTRIES}"
rm -f "${GSS_PRIVATE_DIR}/config.toml" \
"${GSS_PRIVATE_DIR}/.runner_system_id"
rmdir "${GSS_PRIVATE_DIR}"
else
test ! -e "${GSS_PRIVATE_DIR}"
fi
for name in "${GSS_MANAGER_NAME}" "${GSS_PROXY_NAME}"; do
GSS_ABSENCE="$(mktemp)"
if gss_docker inspect "${name}" >"${GSS_ABSENCE}" 2>&1; then
printf 'Scoped container still exists or its name was reused: %s\n' \
"${name}" >&2
exit 1
fi
grep -Eqi 'No such (object|container)' "${GSS_ABSENCE}"
rm -f "${GSS_ABSENCE}"
done
for name in "${GSS_INTERNAL_NETWORK}" "${GSS_OUTBOUND_NETWORK}"; do
GSS_ABSENCE="$(mktemp)"
if gss_docker network inspect "${name}" >"${GSS_ABSENCE}" 2>&1; then
printf 'Scoped network still exists or its name was reused: %s\n' \
"${name}" >&2
exit 1
fi
grep -Eqi 'network .+ not found|[Nn]o such network' "${GSS_ABSENCE}"
rm -f "${GSS_ABSENCE}"
done
while read -r volume; do
test -n "${volume}" || continue
GSS_ABSENCE="$(mktemp)"
if gss_docker volume inspect "${volume}" >"${GSS_ABSENCE}" 2>&1; then
printf 'Scoped volume still exists: %s\n' "${volume}" >&2
exit 1
fi
grep -qi 'no such volume' "${GSS_ABSENCE}"
rm -f "${GSS_ABSENCE}"
done <"${GSS_OWNED_VOLUMES_FILE}"
Before removing a volume, bind it to a container inspection captured during this procedure and confirm that it wasn't in the initial inventory. Runner cache volumes normally carry com.gitlab.gitlab-runner.managed=true; the Runner manager image can create an anonymous home volume without that label. Remove only exact names with provenance from this run. Then check that the scoped containers, networks, private configuration, exact runner ID, and those exact volumes are absent. Preserve every unrelated Docker object.
Keep the private subgroup, project, protected branch, pipeline, job, and artifact until GSS_REVIEW_BY, or delete them after another Owner confirms that the subgroup still contains only this temporary project. GitLab deletion doesn't erase the local Runner token file, so server-side and host-side cleanup are both required.
Review the configuration before changing the workload
Keep runner and Docker-host administration with the platform owner. Before using this configuration for another workload, review its required images, services, credentials and network destinations. Repeat the checkout, artifact upload and access checks after changing the Runner, helper, proxy or task image.
This configuration does not enforce a PID ceiling or force every application job onto this runner. Containers share the host kernel, and the proxy checks cover the specified destinations. Resolve alternative-runner eligibility and choose a host or executor with the required isolation before introducing untrusted application code or production credentials.