docs: sync service guides with current main

Merge pull request 'chore(deps): update renovate/renovate docker tag to v44.147.0' (#114 ) from renovate/renovate-self-update into main
Reviewed-on: #114
2026-10-08 21:23:46 +02:00 · 2026-10-08 19:15:52 +00:00 · 2026-10-08 19:15:52 +00:00 · 2026-10-08 19:15:42 +00:00 · 2026-10-08 19:15:42 +00:00 · 2026-10-08 19:14:10 +00:00
63 changed files with 1530 additions and 243 deletions
@@ -0,0 +1,64 @@
+# CI and deployment
+
+Gitea Actions validates changes, builds the repository's custom images, and can
+deploy selected services to the workstation. CI and production deployment use
+separate workflows. See the [runner and recovery guide](runner/README.md) for
+installation, configuration, and operator commands.
+
+## CI
+
+`workflows/ci.yaml` runs Compose, workflow, shell, formatting, Python and unit
+test, YAML, Dockerfile, and Kubernetes checks. Pull requests and non-main refs
+use the unprivileged `homelab-pr` runner. Main-branch CI uses `homelab`. Tool
+versions are pinned in `workflows/tool-versions.env`.
+
+Compose CI checks every committed Compose file without requiring ignored `.env`
+files. Kubernetes checks validate known schemas; unknown CRDs are skipped.
+
+On main, CI plans builds for the three owned images: `error-pages`,
+`forust-homepage`, and `xdfnx-homepage`. It builds changed inputs or reuses a
+digest from a successful earlier main run. The successful build job publishes a
+release artifact for the exact commit SHA. Pull requests do not publish images.
+
+## Deployment gate
+
+`workflows/deploy.yaml` starts a deployment after successful main CI when the
+`AUTODEPLOY` Actions variable is `true`. Manual dispatch uses the same gate: the
+requested `main` ref or commit must have successful main CI and its matching
+release artifact. A manual dispatch does not bypass validation.
+
+The workflow supports these modes:
+
+- `changed`: select active services changed since the last successful deploy.
+- `full`: select all active services; use this for the first baseline.
+- `plan`: validate and show the selection without applying production resources.
+
+`refresh_images=true` explicitly refreshes mutable third-party Compose tags.
+
+## Selection and rollout
+
+The active markers define automatic deployment. `<service>/active` selects a
+standard Compose file; `<service>/k8s/active` selects Kubernetes resources. Helm
+releases have their own markers in `workflows/deploy-lib.sh`. Service
+dependencies are declared in `deploy-dependencies.json`. Removed resources are
+reported for manual review; the workflow does not prune them automatically.
+
+The workstation controller runs the checked source in a per-SHA worktree. It
+validates configuration, applies Kubernetes and Compose changes in sequence,
+verifies changed Kubernetes workloads, and checks public routes. A durable
+systemd service continues the rollout if the Actions SSH client disconnects.
+The workflow checks the exact CI release before it submits a deployment.
+
+Kubernetes recovery uses captured workload revisions. It does not restore
+ConfigMaps, Secrets, database schemas, or persistent data. Compose recovery is
+manual and does not restore volume data or reverse migrations. Keep backups for
+stateful services. The runner guide documents status, retry, logs, and recovery
+commands.
+
+## Settings
+
+Configure `DEPLOY_HOST`, `DEPLOY_USER`, `DEPLOY_PORT`, and the verified
+`DEPLOY_KNOWN_HOSTS` entry as Actions variables. Keep `DEPLOY_SSH_KEY`,
+`REGISTRY_USERNAME`, and `REGISTRY_PASSWORD` in Actions secrets. The workstation
+also needs its existing registry authentication. Set `AUTODEPLOY=false` until
+automatic production deploys are intended.
@@ -91,7 +91,6 @@ if check_referenced_secrets >"$scratch/secrets.log"; then
  echo 'Secret check accepted a failed manifest render' >&2
  exit 1
 fi
-printf '%s\n' 'Deploy validation regressions passed.'

 # New declared namespaces defer only their own resources during preflight.
 render_selected_resources() {
@@ -132,4 +131,26 @@ if validate_server_resources true 2>"$scratch/undeclared.log"; then
  echo 'Preflight accepted an undeclared missing namespace' >&2
  exit 1
 fi
-printf '%s\n' 'Namespace validation regressions passed.'
+# Count services, not characters in the newline-separated service names.
+compose() {
+  case "$*" in
+    *'config --format json') printf '%s\n' '{"services":{"headscale":{},"headplane":{},"web":{},"init":{"restart":"no"}}}' ;;
+    *'ps --status running --services') printf '%s\n' headscale headplane web ;;
+    *) return 1 ;;
+  esac
+}
+verify_compose_stack example.yaml >"$scratch/compose-count.log"
+grep -qF 'all 3 service(s) running' "$scratch/compose-count.log"
+compose() {
+  case "$*" in
+    *'config --format json') printf '%s\n' '{"services":{"headscale":{},"headplane":{},"web":{}}}' ;;
+    *'ps --status running --services') printf '%s\n' headscale headplane ;;
+    *) return 0 ;;
+  esac
+}
+if verify_compose_stack example.yaml >"$scratch/compose-missing.log"; then
+  echo 'Compose verification accepted a missing service' >&2
+  exit 1
+fi
+grep -qF 'NOT RUNNING: web' "$scratch/compose-missing.log"
+printf '%s\n' 'Deploy validation regressions passed.'
@@ -330,11 +330,11 @@ rollback_workloads() {
 # written straight into a `helm upgrade` command would never be updated: these
 # have to be declared as custom.regex managers in renovate/renovate.json.
 HELM_RELEASES=(
-  "prometheus-stack|prometheus-community/kube-prometheus-stack|prometheus|86.2.3|prometheus-stack/k8s/grafana-values.yaml|prometheus-stack/k8s/active"
+  "prometheus-stack|prometheus-community/kube-prometheus-stack|prometheus|86.3.2|prometheus-stack/k8s/grafana-values.yaml|prometheus-stack/k8s/active"
  "victoria-operator|victoriametrics/victoria-metrics-operator|prometheus|0.68.1|prometheus-stack/k8s/victoria-operator-values.yaml|prometheus-stack/k8s/active"
  "loki|grafana/loki|prometheus|7.3.0|loki/k8s/loki-values.yaml|loki/k8s/active"
  "alloy|grafana/alloy|prometheus|1.12.1|loki/k8s/alloy-values.yaml|loki/k8s/active"
-  "reloader|stakater/reloader|reloader|2.2.17|reloader/k8s/reloader-values.yaml|reloader/k8s/active"
+  "reloader|stakater/reloader|reloader|2.2.18|reloader/k8s/reloader-values.yaml|reloader/k8s/active"
 )

 # "name url" for the Helm repository hosting a chart, empty if unknown.
@@ -827,12 +827,13 @@ stage_verify_k8s() {
 # actually be running.
 verify_compose_stack() {
  local cf="$1"
-  local expected running missing=()
+  local expected running svc missing=() service_count=0
  expected="$(compose "$cf" config --format json | jq -r ' .services | to_entries[] | select(.value.restart != "no") | .key' | sort)" || return 1
  running="$(compose "$cf" ps --status running --services | sort)" || return 1
  [ -n "$expected" ] || return 0
  while IFS= read -r svc; do
    [ -n "$svc" ] || continue
+    service_count=$((service_count + 1))
    # restart:"no" services are allowed to have exited.
    if ! printf '%s\n' "$running" | grep -qx "$svc"; then
      missing+=("$svc")
@@ -843,7 +844,7 @@ verify_compose_stack() {
    compose "$cf" ps --all 2>/dev/null | sed 's/^/    /' || true
    return 1
  fi
-  echo "  all ${#expected} service(s) running"
+  echo "  all $service_count service(s) running"
  return 0
 }

@@ -305,7 +305,6 @@ def build_images(output, report, name, plan):
        if exists:
            print(f'Reuse {name}: inputs unchanged')
            digest = old_digest
-            report['reused'].append(name)
        else:
            print(f'Build {name}', flush=True)
            metadata = Path(docker_config) / 'metadata.json'
@@ -332,14 +331,14 @@ def build_images(output, report, name, plan):
                env=env,
            )
            digest = json.loads(metadata.read_text())['containerimage.digest']
-            report['built'].append(name)
+        if not isinstance(digest, str) or not DIGEST.fullmatch(digest):
+            raise ValueError('Image job returned an invalid digest')
        release['images'][image] = digest
        release['inputs'][image] = inputs
-        if not DIGEST.fullmatch(digest):
-            raise ValueError('Image job returned an invalid digest')
+        report['reused' if exists else 'built'].append(name)
        output.write_text(json.dumps(release, indent=2) + '\n')
        report['current'] = None
-        report['phase'] = 'Release file saved'
+        report['phase'] = 'Image result file saved'
    finally:
        # Cleanup errors must neither leak credentials nor mask the original build error.
        try:
@@ -395,13 +394,17 @@ def build(output, name, plan):
        result = 'success'
    finally:
        lines = [
-            f'## Image release `{os.environ.get("GITHUB_SHA", "unknown")}`',
+            f'## Image build result `{name}`',
+            '',
+            f'- Commit: `{os.environ.get("GITHUB_SHA", "unknown")}`',
            '',
            f'- Result: **{result}**',
            f'- Last stage: {report["phase"]}',
        ]
        if result == 'failure':
-            lines.append('- No release from this build can be deployed. Open the failed step log.')
+            lines.append('- This image job failed. The complete release cannot be published. Open the failed step log.')
+        if result == 'success':
+            lines.append('- This is one image result. The final build job must publish the complete release.')
        if report['current']:
            lines.append(f'- Image at the failure: `{report["current"]}`')
        for title, key in (('Built', 'built'), ('Reused from successful CI', 'reused')):
@@ -14,7 +14,7 @@ ACTIONLINT_VERSION="1.7.7"
 SHELLCHECK_VERSION="0.11.0"
 KUBECONFORM_VERSION="0.8.0"
 PRETTIER_VERSION="3.8.1"
-RUFF_VERSION="0.16.8"
+RUFF_VERSION="0.16.10"
 YAMLLINT_VERSION="1.38.0"
 HADOLINT_VERSION="2.14.0"
 # pip-audit reads the advisory database over the network, so a floating version
@@ -0,0 +1,163 @@
+# Homelab
+
+Configuration for my homelab: Kubernetes manifests, Docker Compose stacks, and the
+Gitea Actions that build and deploy them. Most applications have both deployment
+formats. Headscale and Nextcloud AIO have Compose deployments with Kubernetes
+ingress; the media stack has Compose and Kubernetes routing configuration.
+
+These files contain this lab's domains, IP addresses, storage paths, and private
+registry names. Running them on another machine takes some editing.
+
+## Start here
+
+- [Service list](#services) — what each directory contains.
+- [Deployment workflow](.gitea/README.md) — selection, validation, and recovery.
+- [Repository review](docs/repository-review.md) — findings from the 6 October baseline and their status.
+- [EDU ownership handoff](.gitea/EDU_HANDOFF.md) — the EDU workloads now live in their own repository.
+- [Shared PostgreSQL](postgres/README.md), [Traefik](traefik/README.md), and
+  [cert-manager](cert-manager/README.md) — common dependencies.
+
+## What gets deployed
+
+The `active` files are switches for the deploy workflow, not health indicators.
+
+| File                   | Effect                                                      |
+| ---------------------- | ----------------------------------------------------------- |
+| `<service>/active`     | Include that directory's `compose.yaml` or `compose.yml`.   |
+| `<service>/k8s/active` | Include its Kubernetes manifests or Kustomize overlay.      |
+| Both                   | Run the Compose stack and apply the Kubernetes resources.   |
+| Neither                | Keep the configuration in Git without automatic deployment. |
+
+`shared-compose.yaml`, `client.compose.yaml`, and `renovate-compose.yaml` are
+manual entry points. The deploy script does not discover them.
+
+Kubernetes selection excludes secret files, examples, Helm values, and patches.
+Helm releases listed in `deploy-lib.sh` are upgraded separately. Traefik,
+cert-manager, and CrowdSec have additional bootstrap steps; an `active` marker
+does not install their charts.
+
+The table below describes committed configuration. It does not claim that a
+service is currently healthy or running.
+
+## Services
+
+| Service                                        | Configuration                | Selected by markers |
+| ---------------------------------------------- | ---------------------------- | ------------------- |
+| [AdGuard Home](adguardhome/README.md)          | Kubernetes + Compose         | Kubernetes          |
+| [Authentik](authentik/README.md)               | Kubernetes + Compose         | Kubernetes          |
+| [cert-manager](cert-manager/README.md)         | Kubernetes / Helm            | Manual              |
+| [Cloudflare DDNS](cfddns/README.md)            | Kubernetes + Compose         | Kubernetes          |
+| [Checkmk](checkmk/README.md)                   | Kubernetes + Compose         | Manual              |
+| [Cloudflare Tunnel](cloudflared/README.md)     | Kubernetes / Helm            | Manual              |
+| [File converters](converters/README.md)        | Kubernetes + Compose         | Kubernetes          |
+| [CrowdSec](crowdsec/README.md)                 | Kubernetes / Helm            | Manual              |
+| [Dockmon](dockmon/README.md)                   | Kubernetes + Compose         | Manual              |
+| [Downtify](downtify/README.md)                 | Kubernetes + Compose         | Manual              |
+| [Error pages](errorpages/README.md)            | Kubernetes + Compose         | Kubernetes          |
+| [Gitea](gitea/README.md)                       | Kubernetes + Compose         | Kubernetes          |
+| [Glance](glance/README.md)                     | Kubernetes + Compose         | Manual              |
+| [Headscale](headscale/README.md)               | Compose + Kubernetes routing | Compose, Kubernetes |
+| [Homarr](homarr/README.md)                     | Kubernetes + Compose         | Manual              |
+| [Homepages](homepages/README.md)               | Kubernetes + Compose         | Kubernetes          |
+| [Immich](immich/README.md)                     | Kubernetes + Compose         | Kubernetes          |
+| [Kener](kener/README.md)                       | Kubernetes + Compose         | Manual              |
+| [Loki and Alloy](loki/README.md)               | Kubernetes / Helm            | Kubernetes          |
+| [MeTube](metube/README.md)                     | Kubernetes + Compose         | Kubernetes          |
+| [n8n](n8n/README.md)                           | Kubernetes + Compose         | Manual              |
+| [NetBird](netbird/README.md)                   | Kubernetes + Compose         | Kubernetes          |
+| [NetBox](netbox/README.md)                     | Kubernetes + Compose         | Kubernetes          |
+| [Netronome](netronome/README.md)               | Kubernetes + Compose         | Kubernetes          |
+| [Nextcloud AIO](nextcloud/README.md)           | Compose + Kubernetes routing | Compose, Kubernetes |
+| [Penpot](penpot/README.md)                     | Compose                      | Manual              |
+| [Portainer](portainer/README.md)               | Kubernetes + Compose         | Manual              |
+| [Shared PostgreSQL](postgres/README.md)        | Kubernetes + Compose         | Kubernetes          |
+| [Monitoring stack](prometheus-stack/README.md) | Kubernetes + Compose         | Kubernetes          |
+| [RackPeek](rackpeek/README.md)                 | Kubernetes + Compose         | Kubernetes          |
+| [Reloader](reloader/README.md)                 | Kubernetes / Helm            | Kubernetes          |
+| [Renovate](renovate/README.md)                 | Kubernetes + Compose         | Kubernetes          |
+| [SearXNG](searxng/README.md)                   | Kubernetes + Compose         | Manual              |
+| [Media stack](streaming/README.md)             | Compose + Kubernetes routing | Manual              |
+| [Termix](termix/README.md)                     | Kubernetes + Compose         | Manual              |
+| [Traefik](traefik/README.md)                   | Kubernetes + Compose         | Kubernetes          |
+| [Uptime Kuma](uptime-kuma/README.md)           | Kubernetes + Compose         | Kubernetes          |
+| [Vaultwarden](vaultwarden/README.md)           | Kubernetes + Compose         | Kubernetes          |
+| [3x-ui](vpn/xui/README.md)                     | Kubernetes                   | Kubernetes          |
+
+## Running a Compose stack
+
+Use the service README first. Where a service has an env example, copy it inside
+that service's directory and replace the placeholders. The root `.env.example`
+is an older collection of variables, not a complete configuration for every stack.
+
+For example, from the repository root:
+
+```sh
+cd netbox
+cp .env.example .env
+$EDITOR .env
+docker compose config --quiet
+docker compose up -d
+docker compose ps
+```
+
+Stacks that attach to `proxy` require an existing Docker network of that name and
+an appropriate reverse proxy. Published host ports still work independently of
+Traefik. Check port conflicts before starting an alternative to a Kubernetes
+service: DNS, STUN, and HTTP listeners can share the same host.
+
+`docker compose down` keeps named volumes. Adding `-v` removes them.
+
+## Preparing Kubernetes
+
+The manifests assume Traefik CRDs, cert-manager, and a working storage provisioner.
+PrometheusRule and ServiceMonitor resources also need the Prometheus Operator.
+Replace the lab's hosts and addresses before using the configuration elsewhere.
+
+Create a service's namespace, then prepare its ignored Secret from the example.
+For example:
+
+```sh
+kubectl apply -f netbox/k8s/namespace.yaml
+cp netbox/k8s/secrets.yaml.example netbox/k8s/secrets.yaml
+$EDITOR netbox/k8s/secrets.yaml
+kubectl apply -f netbox/k8s/secrets.yaml
+```
+
+The deploy workflow applies the tracked resources for marked services. Avoid
+applying an entire `k8s/` directory blindly: some directories contain Helm values,
+examples, and alternative routes. For a manual change, apply the selected manifest
+explicitly and check the resulting rollout.
+
+Shared database passwords must agree between the `database` namespace and each
+application's Secret. Updating the PostgreSQL Secret does not change an existing
+role's password; see the database README.
+
+## Local checks
+
+CI pins its tools in `.gitea/workflows/tool-versions.env`. Use the same versions:
+
+```fish
+set tools_dir (bash .gitea/workflows/install-ci-tools.sh)
+set -gx PATH $tools_dir $PATH
+ruff check .
+ruff format --check .
+actionlint -config-file .gitea/actionlint.yaml .gitea/workflows/*.yaml
+.gitea/workflows/sync-renovate-configmap.sh --check
+```
+
+The [workflow README](.gitea/README.md#ci) lists the rest of the checks.
+Structure checks do not establish that local Secrets, mounted files, storage,
+or external services are ready.
+
+## Data and recovery
+
+State lives outside Git: PVCs, Docker volumes, bind mounts, databases, and ignored
+configuration. Keep backups of application data and the keys needed to read it.
+An image rollback does not roll back database migrations or ConfigMap contents.
+
+Many PVCs use the cluster's default StorageClass; monitoring explicitly uses
+`local-path`. Check the PV reclaim policy before deleting a PVC or namespace.
+The manifests do not provide a repository-wide backup schedule.
+
+`incident-archive/` contains past incident notes. `.docs/storage-audit-instruction.md`
+is a planning document, not evidence that NFS has been installed.
@@ -0,0 +1,22 @@
+# AdGuard Home
+
+DNS filtering with a web UI, DNS-over-TLS, and certificates from cert-manager.
+
+The Kubernetes namespace is `adguard`. The workload uses `adguard-pvc` for
+configuration and working data, and mounts the `adguard-certs` TLS Secret.
+The LoadBalancer Service exposes DNS separately from the web ingress.
+
+The Compose stack publishes TCP/UDP 53 and TCP 853 on the host. Prepare `conf/`
+and `certs/` before starting it. Starting both DNS deployments on the same address
+can cause a port conflict.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n adguard
+kubectl get events -n adguard --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,22 @@
+# Authentik
+
+Identity provider with separate server and worker deployments.
+
+Kubernetes connects to the shared PostgreSQL service in `database`. Set
+`AUTHENTIK_DB_PASSWORD` to the same value in both database and application Secrets.
+Keep `AUTHENTIK_SECRET_KEY` with the backups.
+
+Compose uses its own PostgreSQL 15 container and bind-mounted media and templates.
+Its image defaults differ from Kubernetes; check both before an upgrade.
+The worker mounts the Docker socket for Docker outpost management.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n authentik
+kubectl get events -n authentik --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,18 @@
+# cert-manager
+
+Public ACME issuers and an internal certificate authority.
+
+This directory contains chart values and issuer resources, not the controller
+installation. Install the cert-manager chart with CRDs and the settings in
+`k8s/cert-manager-values.yaml` before applying the issuers.
+
+`clusterissuer.yaml` defines staging and production Let's Encrypt issuers.
+They use HTTP-01 through the Traefik ingress class. Public DNS and inbound HTTP
+reachability must work for the requested names before issuance.
+`internal-ca.yaml` bootstraps the internal CA. Keep its private-key Secret backed
+up; the tracked `.crt` is only a public certificate.
+
+This directory has no `k8s/active` marker. Apply the issuer files deliberately;
+`kubectl apply` does not interpret the Helm values file.
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,22 @@
+# Cloudflare DDNS
+
+Updates the lab DNS records when the public address changes.
+
+Kubernetes runs in `default` with host networking and reads `cfddns-secrets`.
+The Compose stack also uses host networking. Configure the API token and domain
+list from the relevant example; keep DNS names consistent with the ingress rules.
+
+`config.json.example` is a separate configuration example. The current Compose
+file does not mount a config.json file. Check configuration against the pinned
+DDNS image when changing between environment and file-based settings.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n default
+kubectl get events -n default --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# Checkmk
+
+Checkmk Raw monitoring site with web and agent-receiver ingress.
+
+The site data lives in `checkmk-sites-pvc` on Kubernetes and the `sites` named
+volume on Compose. The agent receiver has a separate TCP route; enabling the
+web route alone does not expose it.
+
+Prepare the password in the service env or Secret example. Inspect the Checkmk
+container logs during the first site creation.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n checkmk
+kubectl get events -n checkmk --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# Cloudflare Tunnel
+
+A Kubernetes connector for an existing Cloudflare tunnel.
+
+The Deployment runs in `default` and reads its token from the ignored Secret
+created from `k8s/secret.yaml.example`. Create the tunnel and its hostname rules
+in Cloudflare before starting the connector.
+
+There is no Compose file or `k8s/active` marker. Apply the Secret first, then
+`k8s/deployment.yaml` when this tunnel is needed.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n default
+kubectl get events -n default --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,22 @@
+# File converters
+
+ConvertX for server-side conversion and BentoPDF for PDF tools.
+
+ConvertX persists files in `convertx-pvc`; BentoPDF has no persistent volume.
+Kubernetes configuration includes a local `config.yaml.example`, excluded from
+normal deployment. Copy and apply the real ConfigMap separately where required.
+
+Compose publishes ConvertX on host port 9992 as well as attaching it to the
+proxy network. Replace the authentication settings from `.env.example` before
+exposing it outside the lab.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n converters
+kubectl get events -n converters --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,25 @@
+# CrowdSec
+
+Helm values, dashboards, network policy, and a maintenance CronJob.
+
+Install CrowdSec separately using `k8s/crowdsec-values.yaml`; the deploy
+workflow does not have a CrowdSec Helm release entry. There is no `k8s/active`
+marker in this directory.
+
+The LAPI policy and janitor run in `crowdsec`. The dashboard ConfigMaps are in
+`prometheus` for Grafana's sidecar. The janitor has its own ServiceAccount and
+namespace Role. Review its script and schedule before enabling cleanup.
+
+Traefik's values state that enforcement moved to a host firewall bouncer. This
+repository does not install that host component.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n crowdsec
+kubectl get events -n crowdsec --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# Dockmon
+
+Docker management UI that talks to the host Docker daemon.
+
+Both runtimes mount `/var/run/docker.sock`. On Kubernetes the socket belongs
+to the node hosting the pod, so this is not a cluster-wide container manager.
+
+Compose stores application data in a named volume. Kubernetes uses a StatefulSet
+with a volume claim template. Its ServersTransport is specific to the upstream
+connection; keep it with the ingress resources.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n dockmon
+kubectl get events -n dockmon --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,134 @@
+# Repository review (6 October 2026 baseline)
+
+This records the tracked tree at `cc9c3de` and the workstation state observed on
+6 October 2026. It is a historical review, not a current runtime inventory. The
+listed code fixes have since merged into `main`; EDU ownership has moved to the
+separate repository described in [the handoff record](../.gitea/EDU_HANDOFF.md).
+See the [CI and deployment guide](../.gitea/README.md) and
+[runner and recovery guide](../.gitea/runner/README.md) for the current workflow.
+No deployment was performed during the original review.
+
+## Findings at the baseline and current status
+
+| Priority | Finding at the baseline                                                                        | Current status                                                                                                          |
+| -------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
+| High     | Per-file `APPLY_PRUNE=true` could delete resources selected by a shared label.                 | The deploy workflow rejects unsafe pruning before applying resources.                                                   |
+| High     | Compose validation did not resolve the local configuration required at deploy time.            | Preflight resolves the selected Compose configuration before apply.                                                     |
+| Medium   | Secret validation could miss namespace-specific and mounted Secret references.                 | Preflight checks rendered references in their namespaces, including mounted and projected Secrets.                      |
+| Medium   | Compose CI missed manual entry points such as `shared-compose.yaml` and `client.compose.yaml`. | CI checks all tracked Compose files.                                                                                    |
+| Medium   | NetBird Compose referenced missing setup and renderer files.                                   | The setup and renderer files are now present; Compose remains a manual alternative to the active Kubernetes deployment. |
+| Medium   | Glance mounted its CSS from the wrong ConfigMap.                                               | The mount now uses the ConfigMap that contains `user.css`.                                                              |
+| Medium   | The PostgreSQL env example omitted the required NetBox password.                               | The example now includes the required variable.                                                                         |
+| Medium   | The former EDU code had stale Compose variable names and session reliability problems.         | EDU workloads and their fixes moved out of this repository; see the handoff record.                                     |
+| Medium   | AdGuard DoH and SearXNG Compose router expressions used invalid `Host(...)` syntax.            | The router expressions now follow Traefik's rule syntax.                                                                |
+
+Traefik matchers should be combined as `Host(a) || Host(b)`; the rule syntax is
+described in the [Traefik rules documentation](https://doc.traefik.io/traefik/reference/routing-configuration/http/routing/rules-and-priority/).
+The fix retains the DoH path constraint for both hostnames.
+
+The current deploy workflow deliberately rejects the unsafe prune option. It
+does not introduce automatic deletion under a different implementation. The
+baseline finding was a configuration risk, not evidence of a live deletion
+incident.
+
+The former session fix bounded HTTP and Redis calls, validated credentials, set
+a cookie lifetime of two refresh intervals, and marked success only after
+publishing the verified cookie. The service is now owned by the EDU repository;
+see that repository for its current implementation.
+
+The deployment fix extracts required pod Secret references from rendered JSON,
+checks their namespaces, includes init containers, image-pull credentials, and
+mounted/projected Secrets, and honors optional references. Ingress TLS Secrets
+issued by cert-manager are not treated as pre-existing pod prerequisites.
+It checks existence/access, not every key's contents or application validity.
+
+## Live workstation observations
+
+The SSH alias `workstation` is reachable. It has one Ready control-plane node,
+Kubernetes `v1.35.4+k0s`, and a Docker daemon alongside containerd. At inspection,
+no pods were Pending or in another non-running, non-completed phase. This is a
+point-in-time observation, not a complete application health test.
+
+The deployment checkout at `/srv/homelab` is on main commit `2adf17c`, behind the
+reviewed local commit. It has untracked host configuration and a separate
+`userbot/` directory. It was not reset or cleaned.
+
+| Observed difference                                                                                   | Implication                                                                                                                                        |
+| ----------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
+| VictoriaMetrics and vmalert are running; the Prometheus StatefulSet has zero replicas.                | A monitoring migration is already in progress outside committed main. Deploying the old Helm values can overwrite those settings.                  |
+| Homarr, Cloudflared, and Reloader are installed without their current Git active markers.             | Installed services and marker-selected services are different inventories. Missing markers do not establish that a service is stopped.             |
+| Cloudflare DDNS is running in both Docker and Kubernetes.                                             | Confirm which instance should own DNS updates and whether their domain lists overlap before retiring either one. Secret values were not inspected. |
+| Traefik's LoadBalancer exposes port 8080 at `192.168.80.2`.                                           | The direct API listener is deployed; its external reachability was not tested.                                                                     |
+| Default `local-path` has reclaim policy Delete, while many existing PVs have been changed to Retain.  | Current retention is partly live state. Recreating a claim can get a different policy from the old PV.                                             |
+| NetBird, NetBox media/reports/scripts, EDU Redis, Homarr, and VictoriaMetrics have Delete-policy PVs. | Deleting their claims can delete important state. Plan backup and retention changes before namespace cleanup.                                      |
+
+The VictoriaMetrics monitoring trial later merged into `main` in PR #95. The
+first row above records the state before that change. Read
+[`prometheus-stack/README.md`](../prometheus-stack/README.md) for the current
+tracked monitoring configuration; the live observations in this section remain
+a snapshot from 6 October.
+
+## Current recovery limits
+
+The deployment controller and its recovery process changed after this review.
+The current operator workflow is documented in the
+[runner and recovery guide](../.gitea/runner/README.md). The remaining boundaries
+are:
+
+- Kubernetes recovery can restore captured workload revisions. It does not
+  restore ConfigMaps, Secrets, database schemas, or persistent data.
+- Compose recovery is manual. It uses saved resolved configuration, but it does
+  not restore volume data or reverse database migrations.
+- Removed resources require manual review and removal; the deploy workflow does
+  not prune them automatically.
+- Plan mode does not create namespaces. During apply, server validation for new
+  namespaces runs after namespace creation and chart installation; a failed
+  check can leave an empty namespace.
+- Storage policy and backup coverage remain service-specific. Check the live PV,
+  PVC, and backup state before changing stateful workloads.
+
+## Validation
+
+At the review baseline, lint checks passed for Python, shell, workflows, YAML, standard Compose
+files, and Kubernetes resources with available schemas. Kubeconform found 347
+resources in 174 files: 201 valid, 146 skipped CRDs, zero invalid resources.
+That skip count matters: passing schema validation does not validate Traefik rule
+strings or other controller-specific behavior.
+
+Fix validation covers:
+
+- Compose discovery of manual entry points, rejection of required-variable gaps,
+  namespace-scoped and optional Secret references, and API/render failures.
+- NetBird setup idempotence, preservation of existing keys, file permissions,
+  runtime rendering, and rejection of invalid trusted proxy CIDRs.
+- Session refresh success and failure paths, timeouts, cookie expiry, log redaction,
+  missing credentials, and nonpositive refresh intervals.
+- Correct Glance ConfigMap key selection and PostgreSQL initializer/env alignment.
+- YAML and Compose structure for the corrected router rules, compared with the
+  documented Traefik grammar. They were not exercised on the live proxy.
+- Prune rejection before any cluster invocation.
+
+At the time of review, all seven fix branches and the documentation branch
+merged together in a disposable validation worktree. That combined tree passed the
+CI-equivalent local checks, Markdown formatting/lint and link checks, all 35
+Compose structure checks, and 11 Python regression tests plus the shell
+validation regressions. CRD server-side validation and live rollout tests were
+not run.
+
+Runtime tests use fixtures and mocks, not production credentials. Live checks read
+workload metadata, storage policies, chart versions, and container state only.
+They did not read Secret contents or change services.
+
+## Reloader follow-up (baseline)
+
+`fix/reloader-integration` added the active marker and opt-in annotations to
+application Deployments/StatefulSets that consume runtime ConfigMaps or Secrets.
+It corrected AdGuard's misplaced pod-template annotation. The Helm settings use
+annotation-based reloads, keep global auto-reload disabled, and ignore Jobs and
+CronJobs. PostgreSQL workloads are excluded because their credential variables
+and init scripts are only effective on an empty data directory.
+
+The controller was running on the workstation when inspected. The original
+review checked configuration against the pinned chart with Helm rendering and
+manifest validation; it did not change production configuration to provoke a
+test restart or confirm every application's live reload behavior.
@@ -0,0 +1,20 @@
+# Downtify
+
+Download UI with a persistent downloads directory.
+
+Compose stores downloads under `Downtify_downloads/`; Kubernetes uses
+`downtify-downloads-pvc`. The ingress manifests reference shared infrastructure,
+so check certificate and middleware availability before enabling them.
+
+Back up downloads separately if they need to survive storage replacement.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n downtify
+kubectl get events -n downtify --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# Error pages
+
+Static HTTP error pages served by an Nginx image built in CI.
+
+Edit the HTML in `html/`; the Dockerfile copies it into the image.
+Kubernetes exposes `error-pages-service` in `error-pages` for Traefik's error
+middleware. Keep the middleware's namespace and port aligned with that Service.
+
+For a local build, run `docker build -t homelab-error-pages .` from this directory.
+Compose references the private registry image rather than a build context.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n error-pages
+kubectl get events -n error-pages --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,25 @@
+# Gitea
+
+Git hosting with HTTP and a separate SSH route.
+
+Kubernetes uses the shared PostgreSQL service and `gitea-pvc` for repositories
+and application data. Match the Gitea database password with the shared database
+Secret. SSH is routed through Traefik's TCP entrypoint on 2221.
+
+Compose uses a separate PostgreSQL 14 database, bind mounts `gitea-data/` and
+`gitea-db/`, and publishes host port 2221. It is an alternative deployment with
+its own database, not a second frontend for the Kubernetes instance.
+
+Back up repositories, application configuration, and a consistent database dump
+together. Gitea Actions definitions for this repository live in `../.gitea/`.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n gitea
+kubectl get events -n gitea --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,26 @@
+# Glance
+
+Dashboard pages for links, service checks, and Docker containers.
+
+Compose mounts `config/` and `assets/`. The Kubernetes equivalents are embedded
+in `k8s/glance-config.yaml`: `glance-config` holds pages and `glance-assets` holds
+`user.css`. Update both copies when changing shared content.
+
+Kubernetes serves the dashboard under `/glance`. Its pod also mounts the node's
+Docker socket. It references `glance-secrets` for `ADGUARD_PASSWORD`, but there is
+no tracked Secret example; create that Secret in `glance` before starting it.
+Compose expects a local `.env` with the same password.
+
+The pod mounts `user.css` from `glance-assets`, which is the ConfigMap that
+contains that key.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n glance
+kubectl get events -n glance --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,18 @@
+# Headscale
+
+Headscale, Headplane, and a separate web administration UI on Docker.
+
+Kubernetes only provides routes to the Docker host. Update the addresses in
+`k8s/routing/external-service.yaml` if the host moves.
+
+Copy `config/headscale.yaml.example`, `config/headplane.yaml.example`, and
+`config/policy.json.example` to their names without `.example`. Set the public
+server URL, DNS settings, Headplane cookie secret, and Headscale public URL.
+The example URLs are placeholders.
+
+Compose publishes Headscale on 18080, its metrics port on 19090, Headplane on
+13000, and the other UI on 10080. The data volumes store the Headscale database,
+keys, and Headplane state. The embedded DERP configuration needs reachable
+addresses; Compose does not publish its UDP 3478 listener.
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,24 @@
+# Homarr
+
+Dashboard with Kubernetes integration and persistent application state.
+
+Kubernetes uses the `homarr` ServiceAccount and the read-only ClusterRole in
+`k8s/rbac.yaml`. Application data lives in `homarr-pvc`; supply the encryption key
+from `k8s/secrets.yaml.example` before the first start and retain it with backups.
+
+The committed ingress is internal. There is no `k8s/active` marker even though
+manifests exist, so the workflow does not select Homarr automatically.
+
+Compose publishes ports 80 and 81, mounts appdata and the Docker socket, and
+expects a local kubeconfig. Check these host ports against Traefik before use.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n homarr
+kubectl get events -n homarr --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,25 @@
+# Homepages
+
+Two static sites: Forust and xdfnx.
+
+The site sources are in `forust_files/` and `xdfnx_files/`. CI builds each with
+its own Dockerfile and publishes it to the private registry. Kubernetes serves
+the image contents; Compose overlays the source directories as bind mounts.
+
+Both Traefik IngressRoute and Gateway API route manifests are committed.
+Keep their hostnames and backend Services aligned when changing routes.
+Certificate resources cover public and internal hostnames.
+
+Build either site locally with `docker build -f Dockerfile.forust .` or
+`docker build -f Dockerfile.xdfnx .` from this directory.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n homepages
+kubectl get events -n homepages --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,29 @@
+# Immich
+
+Photo library with its own vector-enabled PostgreSQL and machine-learning service.
+
+This database is separate from the shared PostgreSQL instance. Keep the server
+and machine-learning versions aligned when upgrading.
+
+Kubernetes bind-mounts `/mnt/immich/library` from the node. That directory must
+already exist and contain the intended library; moving the pod to a different
+node does not move the files. PostgreSQL and Valkey use StatefulSet storage, and
+the model cache has its own PVC.
+
+Compose reads `UPLOAD_LOCATION` and `DB_DATA_LOCATION` from `.env`. The example
+uses the same library path as Kubernetes. Run one writer against that library;
+do not start both deployments as independent instances over the same files.
+
+Back up the library and a consistent database dump together. The model cache
+can be rebuilt; the photo database cannot.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n immich
+kubectl get events -n immich --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# Kener
+
+Status page with Redis and persistent database and upload directories.
+
+Kubernetes uses `kener-db-pvc`, `kener-uploads-pvc`, and a Redis StatefulSet.
+Compose keeps the corresponding directories in named volumes. Set the signing
+and other credentials from the env or Secret example.
+
+The monitors and route settings live in `k8s/config.yaml` and `k8s/ingress.yaml`.
+There is no active marker for either runtime.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n kener
+kubectl get events -n kener --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -1,6 +1,6 @@
 services:
  kener:
-    image: rajnandan1/kener:4.1.5
+    image: rajnandan1/kener:v4.1.7
    container_name: kener
    restart: unless-stopped
    # ports:
@@ -31,7 +31,7 @@ spec:
    spec:
      containers:
        - name: kener
-          image: rajnandan1/kener:4.1.5
+          image: rajnandan1/kener:v4.1.7
          envFrom:
            - configMapRef:
                name: kener-config
@@ -0,0 +1,16 @@
+# Loki and Alloy
+
+Loki log storage and Alloy collection, both deployed through Helm.
+
+The deploy library lists separate `loki` and `alloy` releases in `prometheus`,
+controlled by this directory's `k8s/active` marker. Chart versions are pinned in
+`deploy-lib.sh`; settings live in `loki-values.yaml` and `alloy-values.yaml`.
+
+Alloy collects Kubernetes logs. Grafana's Loki datasource is configured in the
+monitoring stack. Review Loki retention and storage settings before enabling
+collection on a new cluster.
+
+Check releases with `helm list -n prometheus` and inspect collector logs before
+assuming that an empty Grafana query means there were no events.
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# MeTube
+
+Web downloader behind Traefik.
+
+Compose bind-mounts `MeTube_downloads/` on the host. Kubernetes uses a 20 GiB
+`emptyDir` for `/downloads`: completed downloads disappear when the pod is
+replaced. Download files from the UI promptly if this temporary storage is intended.
+
+Application settings are in `k8s/config.yaml`. Persisting downloads in Kubernetes
+would require changing the volume to a PVC and choosing a storage policy.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n metube
+kubectl get events -n metube --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# n8n
+
+Workflow automation with persistent application and file storage.
+
+Kubernetes keeps application state in `n8n-node-pvc` and files in
+`n8n-files-pvc`; Compose uses `node-data` and `files` named volumes.
+Webhook URLs and proxy settings are committed in the application config.
+
+There is no active marker. Review the URLs before enabling the stack, and retain
+the credential encryption key with the database or application-data backup.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n n8n
+kubectl get events -n n8n --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -1,6 +1,6 @@
 services:
  n8n:
-    image: docker.n8n.io/n8nio/n8n:2.43.0
+    image: docker.n8n.io/n8nio/n8n:2.43.2
    container_name: n8n
    restart: unless-stopped
    environment:
@@ -31,7 +31,7 @@ spec:
    spec:
      containers:
        - name: n8n
-          image: docker.n8n.io/n8nio/n8n:2.43.0
+          image: docker.n8n.io/n8nio/n8n:2.43.2
          envFrom:
            - configMapRef:
                name: n8n-config
@@ -1,12 +1,24 @@
 # NetBird

-Self-hosted NetBird with the combined management, signal, relay, and STUN server. The dashboard and server run behind the repository's existing external Traefik instance on the Docker `proxy` network. Only STUN UDP `3478` is published directly.
+Self-hosted NetBird with the combined management, signal, relay, and STUN server. The dashboard and server run behind Traefik. The Compose configuration uses the external Docker `proxy` network and publishes only STUN UDP `3478` directly.

-The deployment uses SQLite for a single-instance homelab server. The persistent `netbird_data` volume and the datastore encryption key are both required to recover the installation.
+The single-instance server uses SQLite. Back up its data and datastore encryption key together.
+
+## Kubernetes
+
+`k8s/active` selects the Kubernetes deployment. It runs the server and dashboard
+in namespace `netbird`; the server stores SQLite data in `netbird-pvc`. The
+configuration renderer and template are in `k8s/`. Prepare
+`k8s/secrets.yaml` from `k8s/secrets.yaml.example` before the first deploy.
+
+## Compose alternative
+
+The Compose files are available for manual use. There is no root `active` marker,
+so the automatic deploy workflow selects Kubernetes only.

 ## Files

- `compose.yaml`: dashboard and combined server; selected by the marker-driven deploy workflow through `active`.
+- `compose.yaml`: dashboard and combined server; start it manually when using Compose.
 - `config.template.yaml`: non-secret server configuration rendered at startup.
 - `entrypoint.sh`: injects Docker secrets into an in-memory runtime configuration.
 - `client.compose.yaml`: optional host-network peer using a dashboard-generated setup key.
@@ -15,7 +27,7 @@ The deployment uses SQLite for a single-instance homelab server. The persistent

 ## First deployment

-Run these commands on the Docker host before merging the activating branch. The deploy preflight resets tracked files but preserves ignored local state.
+Run these commands on the Docker host before the first Compose start.

 ```bash
 cd /srv/homelab/netbird
@@ -1,96 +1,43 @@
 # NetBox

-NetBox for homelab documentation and visualization. Two runtimes are available:
+Inventory and network documentation with a web process, worker, and Valkey.

-| Runtime | Manifest       | Purpose                                                        |
-| ------- | -------------- | -------------------------------------------------------------- |
-| Docker  | `compose.yaml` | Local stand on `127.0.0.1:8000` (no public exposure)           |
-| k8s     | `k8s/`         | Homelab service on `netbox.forust.xyz` (and the internal name) |
+Kubernetes uses the shared PostgreSQL service at
+`postgres.database.svc.cluster.local:5432`, database and role `netbox`.
+The database and application Secrets must contain the same password.
+Media, reports, scripts, and Valkey have persistent storage.

-Both use the same image (`netboxcommunity/netbox:v4.7-5.1.1`) and Valkey for tasks
-plus a second logical database for caching. The Docker stand keeps its own
-PostgreSQL container, while the k8s deployment uses the shared `database` cluster
-(`postgres.database.svc.cluster.local:5432`, role/database `netbox`); only Valkey
-stays a per-service StatefulSet.
+Compose has its own PostgreSQL container and Valkey instances. It publishes the
+web UI on `127.0.0.1:8000`; its Traefik labels can also expose it while a Docker
+proxy is running. Copy `.env.example` to `.env`, replace the credentials, and run
+`docker compose config --quiet` before starting it.

-## Docker Compose
+## First Kubernetes start

-```bash
-cp .env.example .env
-# replace CHANGE_ME
-docker compose up -d
+Create the namespace and application Secret. Provision the database through the
+shared database initializer on a fresh instance, or create the role and database
+manually on an existing instance; see [PostgreSQL](../postgres/README.md).
+The database NetworkPolicy already includes `netbox`.
+
+Apply the selected application manifests after the database is ready. Startup
+runs schema migrations, so the probes allow a longer first boot. Inspect web and
+worker logs before retrying a slow migration.
+
+## Settings and backup
+
+`configuration/configuration.py` is the Compose settings file. Its Kubernetes
+copy is embedded in `k8s/settings.yaml`; keep them aligned.
+Back up the database and media together. Keep `SECRET_KEY` and
+`API_TOKEN_PEPPER_1`: changing them invalidates sessions or API tokens.
+A container rollback cannot undo a database migration.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n netbox
+kubectl get events -n netbox --sort-by=.metadata.creationTimestamp
 ```

-The UI is available at <http://localhost:8000>. The port is bound to `127.0.0.1`
-intentionally, so this stand is not exposed on the LAN or public interfaces.
-
-The `netbox` service is also attached to the external `proxy` network and carries
-Traefik labels for `netbox.forust.xyz` and `netbox.workstation.internal`. Those
-labels only take effect while the Docker Traefik stack is running; it is currently
-stopped, and the live ingress path in this homelab is the k8s Traefik.
-
-Inspect startup and health with:
-
-```bash
-docker compose ps
-docker compose logs -f netbox
-```
-
-Stop it with `docker compose down`; data is kept in the named volumes
-`netbox-postgres`, `netbox-media-files`, `netbox-reports-files`,
-`netbox-scripts-files` and `netbox-redis-data`.
-
-## Kubernetes
-
-`k8s/` is deployed in the homelab cluster and serves `netbox.forust.xyz` publicly
-plus `netbox.workstation.internal` / `netbox.gigaforust.internal` internally. To
-rebuild it from scratch:
-
-```bash
-# 1. shared PostgreSQL: the password lives in the shared secret, NetBox keeps a copy
-kubectl -n database patch secret postgres-shared-secrets \
-  --type merge -p '{"stringData":{"NETBOX_DB_PASSWORD":"<same value>"}}'
-kubectl -n database exec postgres17-0 -- psql -U postgres -d postgres \
-  -c 'CREATE ROLE netbox LOGIN PASSWORD ...' -c 'CREATE DATABASE netbox OWNER netbox'
-
-# 2. secrets first: the deploy workflow never applies *secret*.yaml
-cp k8s/secrets.yaml.example k8s/secrets.yaml   # replace CHANGE_ME
-kubectl apply -f k8s/secrets.yaml
-
-# 3. manifests
-kubectl apply -f k8s/
-```
-
-The shared cluster is reached at `postgres.database.svc.cluster.local:5432`. Its
-NetworkPolicy (`postgres/k8s/network-policy.yaml`) must list the `netbox` namespace
-or connections are dropped, and `postgres/initdb/01-create-databases.sh` already
-creates the role and database on a fresh data directory. NetBox has no PostgreSQL
-StatefulSet of its own — only `netbox-valkey`.
-
-`netbox.forust.xyz` resolves to this host (`78.98.72.122`) through the `DOMAINS`
-list in the `default/cfddns` secret. cert-manager issues `netbox-prod-tls` with the
-`letsencrypt-prod` issuer, the internal route uses `internal-wildcard-tls`.
-
-Resources are permanent again now that the first-boot migrations are complete:
-the web container reserves `100m`/`512Mi` and is capped at `2` CPU/`2Gi`, the
-worker reserves `50m`/`256Mi` and is capped at `1` CPU/`1Gi`, and Valkey reserves
-`25m`/`64Mi` and is capped at `250m`/`256Mi`. The deliberately generous CPU caps
-leave enough headroom for future schema migrations without letting one process
-consume the whole node.
-
-The first start applies ~810 migrations, each in its own transaction with DDL and
-a commit; every later start is a no-op. The startup probe allows 15 minutes and
-`progressDeadlineSeconds` is 1800 for the same reason. Probes run inside the pod
-and explicitly set `Host: netbox.forust.xyz`; a kubelet `httpGet.host` field would
-replace the probe destination with that public hostname and bypass the pod.
-
-## Secrets
-
- `netbox/.env` (compose) and `netbox/k8s/secrets.yaml` (k8s) are gitignored. Only
-  `.env.example` and `k8s/secrets.yaml.example` are committed.
- `netbox/configuration/configuration.py` is env-driven: hosts, database, Redis and
-  the Django keys all come from the environment, so the same settings file works in
-  both runtimes. The k8s copy lives in the `netbox-settings` ConfigMap
-  (`k8s/settings.yaml`) and must be kept in sync with the file.
- Rotating `SECRET_KEY` invalidates all sessions; rotating `API_TOKEN_PEPPER_1`
-  invalidates every API token.
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,22 @@
+# Netronome
+
+Network monitoring application using the shared PostgreSQL instance on Kubernetes.
+
+Kubernetes reads application settings from its ConfigMap and Secret. Match the
+Netronome role password with `NETRONOME_DB_PASSWORD` in the shared database Secret.
+Its namespace is included in the PostgreSQL NetworkPolicy.
+
+The Compose configuration is a separate deployment; review its local database
+settings and env example before starting it. Keep monitoring history in the
+database backup.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n netronome
+kubectl get events -n netronome --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,17 @@
+# Nextcloud AIO
+
+Nextcloud All-in-One on Docker, with Kubernetes routes to the Docker host.
+
+The master container manages its own child containers through the Docker
+socket. Kubernetes does not run the Nextcloud application; the EndpointSlices
+under `k8s/routing/` point to host services.
+
+Compose publishes the AIO administration interface on 8888. The Apache frontend
+uses host port 11000. `NEXTCLOUD_DATADIR` is `/mnt/nextcloud/ncdata`; prepare that
+storage before first setup and do not change the path casually afterwards.
+
+Use AIO's backup and restore tools for the managed application. Keep the master
+configuration volume and the data directory with the recovery plan. Do not
+remove child containers just because they do not appear as Compose services.
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,12 @@
+# Penpot
+
+A Compose-only design application with frontend, backend, exporter, database, and cache.
+
+There is no active marker or Kubernetes deployment here. Configure the public
+URL and credentials from `.env.example` before starting `compose.yaml`.
+
+Penpot has its own PostgreSQL container. The shared database initializer still
+contains a Penpot role, but this Compose stack does not use it.
+Back up the application assets and database together.
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,21 @@
+# Portainer
+
+Container management UI backed by the host Docker socket.
+
+Kubernetes mounts the node's Docker socket and persists application data in
+`portainer-data-pvc`. This targets Docker on that node, not Kubernetes workloads.
+Compose uses the `portainer_data` volume for its state.
+
+Review initial administrator setup and route access before exposing the UI.
+Neither deployment has an active marker.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n portainer
+kubectl get events -n portainer --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -1,6 +1,6 @@
 services:
  portainer:
-    image: portainer/portainer-ce:2.45.1
+    image: portainer/portainer-ce:2.45.2
    container_name: portainer
    restart: always
    volumes:
@@ -29,7 +29,7 @@ spec:
    spec:
      containers:
        - name: portainer
-          image: portainer/portainer-ce:2.45.1
+          image: portainer/portainer-ce:2.45.2
          ports:
            - containerPort: 9000
          volumeMounts:
@@ -1,35 +1,63 @@
 # Shared PostgreSQL

-This directory contains the shared PostgreSQL 17 deployment for Authentik,
-Gitea, NetBox, Netronome, and Statuspage. It creates one database and one login role
-per service. Per-service standalone databases were removed after the
-migration (Sep 2026); Penpot stays on its own compose PostgreSQL (archived,
-not part of the shared instance).
+PostgreSQL 17 for the Kubernetes deployments of Authentik, Gitea, NetBox, and Netronome.

-## Compatibility baseline
+The server runs in `database` as StatefulSet `postgres17`, with data in
+`postgres17-data`. Applications connect to
+`postgres.database.svc.cluster.local:5432`. The NetworkPolicy allows only the
+listed application namespaces; add a new consumer there as well as provisioning
+its database.

-| Service    | Current application | Shared PostgreSQL 17                   |
-| ---------- | ------------------- | -------------------------------------- |
-| Authentik  | 2025.10.x           | Supported (Authentik requires 14+)     |
-| Gitea      | 1.27.3              | Supported (Gitea requires 12+)         |
-| NetBox     | 4.7.x               | Supported (NetBox 4.x requires 13+)    |
-| Netronome  | 0.14.0              | Supported (upstream's example uses 17) |
-| Statuspage | custom              | Supported                              |
+## Initialization

-A major-version change must use a logical dump/restore; changing only the
-image tag while keeping a data directory is not supported.
+`initdb/01-create-databases.sh` creates roles and databases on an empty data
+directory. The Kubernetes copy is embedded in `k8s/postgres.yaml`.
+It also provisions Penpot and Statuspage roles, even though those are not active
+consumers in the current Kubernetes manifests.

-For Compose, copy `.env.example` to `.env`, set all passwords, and start it with
-`docker compose -f shared-compose.yaml up -d`. This file is intentionally not
-named `compose.yaml`, so the repository deploy workflow does not start a second
-database accidentally.
-Applications that use this database must also join that external network and use
-`homelab-postgres:5432`.
+The initializer requires every listed password. Prepare `k8s/secrets.yaml` from
+the example before applying the StatefulSet. Existing application Secrets keep
+copies of their own database passwords; they must match the corresponding role.

-For Kubernetes, create `k8s/secrets.yaml` from the example before applying the
-manifests. The `k8s/active` marker makes the normal deploy workflow include the
-namespace, StatefulSet, ConfigMap, and NetworkPolicy. Applications use
-`postgres.database.svc.cluster.local:5432`.
-Migrate each existing database with a tested logical dump/restore before
-switching an application. Do not reuse a PostgreSQL 14 or 17 data directory
-with PostgreSQL 15.
+The init scripts do not run again when an existing data directory is mounted.
+Changing a Secret does not rotate the PostgreSQL role password. Rotate the role
+with SQL and update the application Secret together.
+
+## Compose alternative
+
+From this directory:
+
+```sh
+cp .env.example .env
+$EDITOR .env
+docker compose -f shared-compose.yaml config --quiet
+docker compose -f shared-compose.yaml up -d
+```
+
+The example includes `NETBOX_DB_PASSWORD`; fill it and every other required
+password before starting the stack.
+This stack creates the `homelab-database` Docker network and the
+`homelab-postgres` container. Compose applications need to join that network
+explicitly to use it; several committed Compose stacks use their own databases.
+
+The filename is intentional: the automatic deploy discovery does not start this
+stack just because the Kubernetes database is active.
+
+## Backup and upgrades
+
+Keep database dumps and role definitions, including ownership and grants.
+Take a logical backup before changing a major PostgreSQL version. A new image
+tag over the existing data directory is not a major-version migration.
+Test restores separately before changing application connection settings.
+Immich uses its own vector-enabled database and is outside this shared instance.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n database
+kubectl get events -n database --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,32 @@
+# Monitoring stack
+
+The Kubernetes stack provides Grafana, Alertmanager, VictoriaMetrics, VMAgent,
+and vmalert. The `k8s/active` marker selects the stack. The
+`kube-prometheus-stack` Helm release installs Grafana, Alertmanager, the
+Prometheus Operator, and related components. Its Prometheus server is configured
+with zero replicas while VMAgent collects metrics and writes them to the
+single-node VictoriaMetrics instance.
+
+The `victoria-operator` Helm release converts selected Prometheus Operator
+`ServiceMonitor` resources into `VMServiceScrape` resources. VMAgent selects
+those scrapes across namespaces and writes to VictoriaMetrics. vmalert evaluates
+the rule ConfigMap and sends alerts to the stack's Alertmanager. See the
+[Kubernetes monitoring notes](k8s/README.md) for application metrics and
+validation commands.
+
+The chart versions are pinned in `.gitea/workflows/deploy-lib.sh`. The tracked
+`k8s/grafana-values.yaml` contains the Helm values for the stack. Create the
+`grafana-admin` and `alertmanager-config` Secrets from the examples in `k8s/`;
+keep their credentials out of the values file. Persistent volumes store data for
+Prometheus, Grafana, Alertmanager, and VictoriaMetrics. Check the PVCs and
+backups before changing storage. VictoriaMetrics currently retains 30 days of
+data.
+
+A separate Compose configuration is present for manual use. There is no root
+`active` marker, so the automatic deploy workflow does not select it.
+
+The deploy workflow does not remove resources when manifests are deleted. For a
+rollback of application-metrics changes, follow the explicit cleanup steps in
+the [Kubernetes monitoring notes](k8s/README.md).
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,20 @@
+# RackPeek
+
+Rack inventory UI behind Traefik.
+
+Kubernetes stores configuration in `rackpeek-pvc`. The Compose alternative uses
+its own data mount. Keep rack descriptions and inventory data in the backup.
+
+Public and internal certificates and routes are in `k8s/`. There are no tracked
+Secret examples for this service.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n rackpeek
+kubectl get events -n rackpeek --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,56 @@
+# Reloader
+
+Restarts opted-in workloads when the ConfigMaps or Secrets they consume change.
+The deploy workflow upgrades the `reloader` Helm release in namespace `reloader`;
+`k8s/active` enables it. The chart version is pinned in `deploy-lib.sh`.
+
+## Workload integration
+
+Put this annotation on the Deployment or StatefulSet metadata:
+
+```yaml
+metadata:
+  annotations:
+    reloader.stakater.com/auto: "true"
+```
+
+The annotation belongs to the workload, not `spec.template.metadata`.
+Reloader discovers references in environment variables and mounted volumes.
+This covers startup-only settings and ConfigMaps or Secrets mounted with `subPath`.
+See the [upstream usage guide](https://github.com/stakater/Reloader/blob/v1.4.22/README.md#usage).
+
+The application manifests opt in workloads including AdGuard's TLS files,
+NetBird, both NetBox processes, and the password-protected Valkey servers.
+Inactive services have the same annotations ready for later activation.
+
+## Controller policy
+
+The controller watches all namespaces but only restarts annotated workloads.
+It uses the `annotations` reload strategy, so changes trigger a pod-template
+annotation rather than injecting extra environment variables.
+
+Jobs and CronJobs are excluded: their next execution reads current configuration.
+PostgreSQL is intentionally not opted in. Its password variables and init scripts
+apply to first initialization; restarting an existing database does not rotate
+roles or rerun those scripts. Rotate database credentials with SQL and update the
+clients' Secrets together.
+
+Helm-managed monitoring components already have their own configuration reload
+paths; Traefik watches its file-provider configuration. They are not globally
+opted in. The controller does not react to files in PVCs or changes to external
+services unless a watched ConfigMap or Secret changes.
+
+## Verify
+
+```sh
+kubectl -n reloader rollout status deployment/reloader-reloader
+kubectl -n reloader logs deployment/reloader-reloader --since=10m
+kubectl -n netbird get deployment netbird-server-deployment \
+  -o jsonpath='{.metadata.annotations.reloader\.stakater\.com/auto}'
+```
+
+A changed configuration can briefly interrupt a single-replica service, especially
+one using `Recreate`. Installing annotations does not validate the configuration
+or migrate database data. Keep changes to shared Secrets coordinated across consumers.
+
+See the [repository README](../README.md) for deployment selection.
@@ -1,101 +1,51 @@
-# Renovate for Gitea
+# Renovate

-Renovate runs as a Kubernetes CronJob and creates container image update pull
-requests in Gitea. It does not deploy changes itself.
+Container and chart dependency updates for the Gitea repository.

-## Kubernetes
+The Kubernetes CronJob runs in `renovate` every six hours with overlapping
+CronJob executions forbidden. Prepare the bot PAT from the Secret example.
+Give the dedicated Gitea user access to the repositories it should update.

-Create a dedicated Gitea user named `renovate-bot`, create a repository access
-token, and grant it repository read/write plus issue read/write permissions.
-Add `read:packages` if Renovate must inspect private Gitea registry images.
-
-Create the ignored Secret locally; never commit the PAT:
+`renovate.json` is the source configuration. The ConfigMap is a generated copy:

 ```sh
-cp renovate/k8s/secrets.yaml.example renovate/k8s/secrets.yaml
-$EDITOR renovate/k8s/secrets.yaml
-kubectl apply -f renovate/k8s/namespace.yaml
-kubectl apply -f renovate/k8s/secrets.yaml
-kubectl apply -f renovate/k8s/configmap.yaml
-kubectl apply -f renovate/k8s/cronjob.yaml
+.gitea/workflows/sync-renovate-configmap.sh
+.gitea/workflows/sync-renovate-configmap.sh --check
 ```

-The `renovate/k8s/active` marker makes the normal deployment workflow include
-the namespace, ConfigMap, and CronJob. The Secret is intentionally excluded
-from Git and must be applied separately after every new cluster.
+Run those commands from the repository root. The `renovate-ci` workflow checks
+that the generated configuration agrees with the source.

-Run it immediately instead of waiting for the six-hour schedule.
+## Run manually

-Two options, both use the same `renovate/renovate.json`:
+From the repository root:
+
+```fish
+kubectl create job --from=cronjob/renovate renovate-manual-(date +%s) -n renovate
+kubectl get jobs,pods -n renovate
+```
+
+Alternatively use the `renovate-run` Actions workflow. It reads the image tag
+from the CronJob and accepts repository, log-level, and dry-run inputs. Actions
+requires `RENOVATE_TOKEN`; `RENOVATE_GITHUB_COM_TOKEN` is optional.
+The Actions concurrency group and the CronJob policy are separate, so avoid
+starting both against the same repository at once.
+
+For Compose, copy `.env.example` to `.env` in this directory and run
+`docker compose -f renovate-compose.yaml run --rm renovate`. That file is a
+manual entry point and is not selected by the deploy workflow.
+
+The config also tracks chart versions in `deploy-lib.sh` and tool versions in
+`.gitea/workflows/tool-versions.env`. Renovate opens pull requests; the normal CI and deploy
+workflows handle changes after merge.
+
+## Inspect
+
+From the repository root:

 ```sh
-kubectl create job --from=cronjob/renovate renovate-manual-$(date +%s) -n renovate
+kubectl get pods,svc,pvc -n renovate
+kubectl get events -n renovate --sort-by=.metadata.creationTimestamp
 ```

-or the `renovate-run` Actions workflow (Actions tab → `renovate-run` →
-Run workflow). It runs the same image as the CronJob on the self-hosted runner
-via Docker — the tag is read out of `renovate/k8s/cronjob.yaml` at run time
-rather than hardcoded, so the two cannot drift apart. Required Actions secrets
-(repo or org settings):
-
- `RENOVATE_TOKEN` — renovate-bot PAT (repository + issue read/write).
- `RENOVATE_GITHUB_COM_TOKEN` — optional, for changelogs and GitHub rate limits.
-
-Inputs: `repositories` (default `forust/homelab`), `log_level`
-(`info`/`debug`). Only one run at a time (concurrency group
-`renovate-run`), same as the CronJob `Forbid` policy.
-
-Inspect runs with:
-
-```sh
-kubectl get cronjob,jobs,pods -n renovate
-kubectl logs -n renovate job/<job-name>
-```
-
-`RENOVATE_GITHUB_COM_TOKEN` is optional but recommended for changelogs and
-GitHub API rate limits. Set it in the Kubernetes Secret if available.
-
-## Compose
-
-Copy `.env.example` to `.env`, set the PAT, and run:
-
-```sh
-docker compose -f renovate-compose.yaml run --rm renovate
-```
-
-The Compose file is intentionally named `renovate-compose.yaml`, so the
-repository's automatic deployment discovery does not start it accidentally.
-
-## Configuration
-
-`renovate/renovate.json` is the single source of truth. The Compose file and the
-`renovate-run` workflow mount that file directly.
-
-A ConfigMap cannot read from the repository, so the CronJob needs the config
-inlined. `renovate/k8s/configmap.yaml` is therefore a **generated** copy:
-
-```sh
-.gitea/workflows/sync-renovate-configmap.sh          # regenerate after editing
-.gitea/workflows/sync-renovate-configmap.sh --check  # fail if out of date
-```
-
-The `renovate-ci` workflow runs the `--check` form on every PR and push, so a
-config edit that forgets to regenerate the ConfigMap cannot be merged.
-
-Beyond images, `customManagers` in the config track:
-
- Helm chart versions pinned in `.gitea/workflows/deploy-lib.sh`. The built-in
-  `helmv3` manager only reads `Chart.yaml` and `helm-values` only reads values
-  files, so neither sees a version written into a `helm upgrade` command —
-  these are declared as `custom.regex` managers against the `helm` datasource.
- CI linter versions in `.gitea/workflows/tool-versions.env`.
-
-The Renovate image tag is deliberately _not_ in `tool-versions.env`:
-`renovate/k8s/cronjob.yaml` owns it, and the workflows read it from there.
-
-## How updates flow
-
-Renovate scans both `compose.yaml` files and Kubernetes manifests, opens a
-branch and PR with image tag changes, and waits for CI. After merge, the
-existing deployment workflow applies Kubernetes changes or redeploys Compose
-stacks. Renovate never updates running workloads directly.
+See the [repository README](../README.md) for deployment selection.
@@ -19,7 +19,7 @@ spec:
          restartPolicy: Never
          containers:
            - name: renovate
-              image: renovate/renovate:44.140.0
+              image: renovate/renovate:44.147.0
              env:
                - name: RENOVATE_PLATFORM
                  value: gitea
@@ -2,7 +2,7 @@ services:
  renovate:
    # Kept in step with renovate/k8s/cronjob.yaml by the "renovate self-update"
    # package rule in renovate/renovate.json.
-    image: renovate/renovate:44.136.0
+    image: renovate/renovate:44.147.0
    container_name: renovate
    restart: "no"
    env_file:
@@ -0,0 +1,22 @@
+# SearXNG
+
+Search frontend with a separate Valkey cache.
+
+Kubernetes keeps the application settings in a ConfigMap and starts Valkey as a
+StatefulSet. Set the secret from the example before exposing the search endpoint.
+There is no active marker.
+
+Compose expects local configuration under `core-config/`, which is ignored.
+Prepare it before starting the stack; a container image alone does not supply
+this lab's settings.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n searxng
+kubectl get events -n searxng --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,18 @@
+# Media stack
+
+Docker services for playback, requests, library management, and downloads.
+
+Compose runs Jellyfin, Jellyseerr, Sonarr, Radarr, Prowlarr, qBittorrent, and the
+other services declared in the file. Kubernetes only routes to host endpoints;
+update `k8s/routing/external-service.yaml` when the Docker host or ports change.
+
+Prepare the paths, user/group IDs, and credentials from `.env.example`. Service
+configuration and media/download directories are bind mounts. Preserve their
+permissions when moving data, and keep the application databases with backups.
+
+Review device mounts for hardware acceleration before starting on another host.
+The Compose and Kubernetes routing files have no active markers, so automatic
+deploys do not select this stack. Start the Compose project or apply its routing
+resources manually when needed.
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,20 @@
+# Termix
+
+Terminal and SSH connection manager with persistent application data.
+
+Kubernetes stores state in `termix-pvc`; Compose mounts `termix-data/`.
+The application config and routes are committed separately under `k8s/`.
+
+There is no active marker. Review access control and retain the application data
+needed to recover saved connections before enabling it.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n termix
+kubectl get events -n termix --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,24 @@
+"""Keep unit-test workflow commands out of the real CI job files."""
+
+import os
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+CI_COMMAND_FILES = ('GITHUB_STEP_SUMMARY', 'GITHUB_OUTPUT', 'GITHUB_ENV', 'GITHUB_PATH', 'GITHUB_STATE')
+
+
+class IsolatedCITestCase(unittest.TestCase):
+    def setUp(self):
+        super().setUp()
+        directory = tempfile.TemporaryDirectory(prefix='homelab-test-ci-')
+        self.addCleanup(directory.cleanup)
+        paths = {}
+        for variable in CI_COMMAND_FILES:
+            path = Path(directory.name) / variable
+            path.touch()
+            paths[variable] = str(path)
+        environment = patch.dict(os.environ, paths)
+        environment.start()
+        self.addCleanup(environment.stop)
@@ -0,0 +1,60 @@
+"""Run the real unit tests with external CI files and detect leaked writes."""
+
+import os
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+from unittest.mock import patch
+
+from ci_test_case import CI_COMMAND_FILES, IsolatedCITestCase
+
+
+class CIOutputIsolationTests(IsolatedCITestCase):
+    def test_all_command_files_are_private_and_environment_is_restored(self):
+        with tempfile.TemporaryDirectory() as scratch:
+            external = {variable: str(Path(scratch) / variable) for variable in CI_COMMAND_FILES}
+            for path in external.values():
+                Path(path).write_text('external CI file\n')
+            with patch.dict(os.environ, external):
+                probe = IsolatedCITestCase()
+                probe.setUp()
+                private = []
+                try:
+                    for variable in CI_COMMAND_FILES:
+                        self.assertNotEqual(os.environ[variable], external[variable])
+                        path = Path(os.environ[variable])
+                        private.append(path)
+                        path.write_text('test-only command\n')
+                finally:
+                    probe.doCleanups()
+                for variable in CI_COMMAND_FILES:
+                    self.assertEqual(os.environ[variable], external[variable])
+                    self.assertEqual(Path(external[variable]).read_text(), 'external CI file\n')
+                self.assertTrue(all(not path.exists() for path in private))
+
+    def test_unit_suite_preserves_external_ci_files(self):
+        tests = Path(__file__).resolve().parent
+        modules = sorted(p.stem for p in tests.glob('test_*.py') if p.name != Path(__file__).name)
+        with tempfile.TemporaryDirectory() as scratch:
+            environment = os.environ.copy()
+            environment['PYTHONPATH'] = str(tests) + os.pathsep + environment.get('PYTHONPATH', '')
+            expected = {}
+            for variable in CI_COMMAND_FILES:
+                path = Path(scratch) / variable
+                content = f'external {variable}\n'
+                path.write_text(content)
+                environment[variable] = str(path)
+                expected[path] = content
+            result = subprocess.run(  # noqa: S603 -- Run local test modules with the current Python interpreter.
+                [sys.executable, '-m', 'unittest', *modules, '-q'],
+                cwd=tests.parent,
+                env=environment,
+                capture_output=True,
+                text=True,
+                check=False,
+                timeout=60,
+            )
+            self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
+            for path, content in expected.items():
+                self.assertEqual(path.read_text(), content, f'Unit tests wrote to external {path.name}')
@@ -9,6 +9,8 @@ import unittest
 from pathlib import Path
 from unittest.mock import patch

+from ci_test_case import IsolatedCITestCase
+
 ROOT = Path(__file__).resolve().parents[1]


@@ -34,7 +36,7 @@ def release(sha='a' * 40):
    }


-class ReleaseGateTests(unittest.TestCase):
+class ReleaseGateTests(IsolatedCITestCase):
    def test_release_rejects_wrong_sha_missing_images_and_mutable_tags(self):
        for mutation in ('sha', 'missing', 'tag'):
            data = release()
@@ -91,8 +93,9 @@ class ReleaseGateTests(unittest.TestCase):
                api.release({'id': 1, 'head_sha': 'a' * 40})


-class SelectionTests(unittest.TestCase):
+class SelectionTests(IsolatedCITestCase):
    def setUp(self):
+        super().setUp()
        self.scratch = tempfile.TemporaryDirectory()
        self.addCleanup(self.scratch.cleanup)
        self.repo = Path(self.scratch.name)
@@ -171,7 +174,7 @@ class SelectionTests(unittest.TestCase):
        self.assertEqual(result['selected']['k8s'], ['one', 'postgres', 'two'])


-class ComposeConfigurationTests(unittest.TestCase):
+class ComposeConfigurationTests(IsolatedCITestCase):
    def test_pin_preserves_project_volumes_paths_and_previous_image(self):
        with tempfile.TemporaryDirectory() as scratch:
            root = Path(scratch)
@@ -305,7 +308,7 @@ class ComposeConfigurationTests(unittest.TestCase):
                )


-class ControllerTests(unittest.TestCase):
+class ControllerTests(IsolatedCITestCase):
    def test_completed_stage_cannot_apply_again(self):
        with tempfile.TemporaryDirectory() as scratch:
            directory = Path(scratch)
@@ -10,10 +10,11 @@ import zipfile
 from pathlib import Path
 from unittest.mock import Mock, patch

+from ci_test_case import IsolatedCITestCase
 from test_cicd import ROOT, controller, release, release_module


-class ArtifactTests(unittest.TestCase):
+class ArtifactTests(IsolatedCITestCase):
    def test_archive_rejects_nested_or_extra_files(self):
        api = object.__new__(release_module.Gitea)
        api.base = 'https://example.test/api/v1/repos/a/b'
@@ -103,7 +104,7 @@ class ArtifactTests(unittest.TestCase):
            self.assertEqual(json.loads((root / 'error-pages.json').read_text())['sha'], 'e' * 40)


-class DurableRunTests(unittest.TestCase):
+class DurableRunTests(IsolatedCITestCase):
    def test_duplicate_start_only_reattaches(self):
        with tempfile.TemporaryDirectory() as scratch:
            state = Path(scratch)
@@ -203,7 +204,7 @@ class DurableRunTests(unittest.TestCase):
            self.assertEqual(json.loads((directory / 'status.json').read_text())['state'], 'failure')


-class FailureSummaryTests(unittest.TestCase):
+class FailureSummaryTests(IsolatedCITestCase):
    def test_build_failure_keeps_progress_and_does_not_expose_exception_text(self):
        with tempfile.TemporaryDirectory() as scratch:
            summary = Path(scratch) / 'summary.md'
@@ -225,6 +226,77 @@ class FailureSummaryTests(unittest.TestCase):
            self.assertIn('xdfnx-homepage', content)
            self.assertNotIn('private value', content)

+    def test_invalid_digest_is_not_reported_as_a_completed_image(self):
+        for digest in ('invalid-private-metadata', None, ['invalid']):
+            with self.subTest(digest=digest), tempfile.TemporaryDirectory() as scratch:
+                root = Path(scratch)
+                summary = root / 'summary.md'
+                name = 'error-pages'
+                context, dockerfile = release_module.IMAGES[name]
+                plan = {
+                    'sha': 'a' * 40,
+                    'targets': [
+                        {
+                            'name': name,
+                            'context': context,
+                            'dockerfile': dockerfile,
+                            'inputs': 'c' * 64,
+                            'reuse_digest': None,
+                        }
+                    ],
+                }
+
+                def fake_command(*args, digest=digest, **_kwargs):
+                    if args[:3] == ('docker', 'buildx', 'build'):
+                        Path(args[args.index('--metadata-file') + 1]).write_text(
+                            json.dumps({'containerimage.digest': digest})
+                        )
+                    return ''
+
+                with (
+                    patch.dict(
+                        os.environ,
+                        {
+                            'GITHUB_STEP_SUMMARY': str(summary),
+                            'GITHUB_SHA': 'a' * 40,
+                            'REGISTRY_USERNAME': 'test',
+                            'REGISTRY_PASSWORD': 'placeholder',
+                        },
+                    ),
+                    patch.object(release_module, 'checked_plan', return_value=plan),
+                    patch.object(release_module.Path, 'home', return_value=root),
+                    patch.object(release_module, 'command', side_effect=fake_command),
+                    patch.object(subprocess, 'run', return_value=subprocess.CompletedProcess([], 0)),
+                    self.assertRaisesRegex(ValueError, 'invalid digest'),
+                ):
+                    release_module.build(root / 'image.json', name, root / 'plan.json')
+                self.assertFalse((root / 'image.json').exists())
+                content = summary.read_text()
+                self.assertIn('**failure**', content)
+                self.assertIn('### Built\n- None', content)
+                self.assertIn('### Completed image digests\n- None', content)
+                self.assertNotIn('invalid-private-metadata', content)
+
+    def test_successful_image_result_does_not_claim_complete_release(self):
+        def complete_image(_output, report, _name, _plan):
+            report.update(phase='Image result file saved', built=['error-pages'])
+            report['images']['gcr.forust.xyz/forust/error-pages'] = 'sha256:' + 'b' * 64
+
+        with (
+            patch.dict(os.environ, {'GITHUB_SHA': 'a' * 40}),
+            patch.object(
+                release_module,
+                'build_images',
+                side_effect=complete_image,
+            ),
+        ):
+            release_module.build(Path('unused.json'), 'error-pages', Path('unused-plan.json'))
+        content = Path(os.environ['GITHUB_STEP_SUMMARY']).read_text()
+        self.assertIn('## Image build result `error-pages`', content)
+        self.assertIn('Commit: `' + 'a' * 40 + '`', content)
+        self.assertIn('final build job must publish the complete release', content)
+        self.assertNotIn('## Image release', content)
+
    def test_deploy_failure_reports_completed_apply_and_rollback_result(self):
        with tempfile.TemporaryDirectory() as scratch:
            state = Path(scratch)
@@ -261,7 +333,7 @@ class FailureSummaryTests(unittest.TestCase):
            self.assertIn('Compose requires manual recovery', content)


-class InstallerTests(unittest.TestCase):
+class InstallerTests(IsolatedCITestCase):
    def test_version_comparison_is_exact_without_network_or_host_packages(self):
        with tempfile.TemporaryDirectory() as scratch:
            root = Path(scratch)
@@ -3,10 +3,10 @@
 import json
 import os
 import tempfile
-import unittest
 from pathlib import Path
 from unittest.mock import Mock, call, patch

+from ci_test_case import IsolatedCITestCase
 from test_cicd import release, release_module


@@ -26,7 +26,7 @@ def plan_data(changed):
    return {'sha': 'a' * 40, 'targets': targets}


-class MatrixTests(unittest.TestCase):
+class MatrixTests(IsolatedCITestCase):
    def test_no_change_one_image_all_images_and_missing_baseline(self):
        for changed in (set(), {'error-pages'}, set(release_module.IMAGES)):
            with self.subTest(changed=changed), tempfile.TemporaryDirectory() as scratch:
@@ -7,11 +7,14 @@ import tempfile
 import unittest
 from pathlib import Path

+from ci_test_case import IsolatedCITestCase
+
 ROOT = Path(__file__).resolve().parents[1]


-class NetbirdRuntimeTests(unittest.TestCase):
+class NetbirdRuntimeTests(IsolatedCITestCase):
    def setUp(self):
+        super().setUp()
        self.temp = tempfile.TemporaryDirectory()
        self.addCleanup(self.temp.cleanup)
        self.root = Path(self.temp.name)
@@ -0,0 +1,33 @@
+# Traefik
+
+Ingress for HTTP, gRPC, TCP, and UDP services, with public and internal TLS.
+
+Kubernetes uses the Helm settings in `k8s/traefik-values.yaml`. The deploy
+library applies supporting resources in this directory but does not install or
+upgrade the Traefik chart. Bootstrap the chart and CRDs separately.
+
+The LoadBalancer address is set to `192.168.80.2`. Change it for another network.
+Entrypoints include web traffic, Gitea SSH, NetBird STUN, and other lab protocols.
+Public certificates come from cert-manager; internal certificates use the lab CA.
+The file provider reads `traefik-dynamic` through an additional volume and flags.
+
+## API access
+
+The committed chart values enable `api.insecure` and expose TCP 8080 through the
+LoadBalancer for Homarr integration. That listener has no Traefik authentication.
+Its reachability depends on external network controls. Review those controls
+before deploying these values outside the trusted network.
+
+The normal dashboard IngressRoute is a separate path; protecting that route does
+not protect the direct port 8080 listener.
+
+## Compose alternative
+
+Compose mounts static and dynamic config, certificates, ACME state, and the
+Docker socket. It needs the external `proxy` network. Local file-server routing
+and TLS files have `.example` templates; copy only the ones needed for the host.
+
+Keep ACME state and private keys with backups. Changing ingress values can affect
+every service at once, so inspect routes and entrypoints after an upgrade.
+
+See the [repository README](../README.md) for deployment selection.
@@ -1,6 +1,6 @@
 services:
  traefik:
-    image: traefik:v3.7.13
+    image: traefik:v3.7.14
    container_name: traefik
    restart: unless-stopped
    command:
@@ -3,7 +3,7 @@ hostNetwork: false
 image:
  registry: docker.io/library
  repository: traefik
-  tag: v3.7.13
+  tag: v3.7.14

 securityContext:
  capabilities:
@@ -0,0 +1,22 @@
+# Uptime Kuma
+
+Service checks, status pages, and Prometheus metrics.
+
+Kubernetes keeps state in `uptime-kuma-pvc` and exposes metrics through a
+ServiceMonitor. The metrics credentials come from the local Secret example.
+`alerts.yaml` adds Prometheus rules; a running Kuma UI alone does not establish
+that Prometheus is scraping it.
+
+Compose stores state in `data/`. Back up that application database and verify
+notification delivery after restoring it.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n uptime-kuma
+kubectl get events -n uptime-kuma --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,23 @@
+# Vaultwarden
+
+Password vault server with persistent data and public/internal ingress.
+
+Kubernetes uses `vaultwarden-pvc` and the public URL from a ConfigMap.
+Compose has a separate data volume. Preserve the database, attachments, and keys
+as part of the same backup.
+
+The env example only sets `DOMAIN`; there is no tracked administrator Secret
+example. Configure any administrator token separately and keep it out of Git. Verify
+sign-in and client synchronization after any update. Do not use a successful
+container restart as the only restore check.
+
+## Inspect
+
+From the repository root:
+
+```sh
+kubectl get pods,svc,pvc -n vaultwarden
+kubectl get events -n vaultwarden --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../README.md) for deployment selection.
@@ -0,0 +1,5 @@
+# VPN services
+
+[x-ui](xui/README.md) contains the Kubernetes deployment for 3x-ui. This directory
+has no shared Compose stack. Active markers are checked at each service's `k8s/`
+level, including nested paths.
@@ -0,0 +1,18 @@
+# 3x-ui
+
+Kubernetes deployment for the 3x-ui management panel in namespace `xui`.
+The `k8s/active` marker includes it in normal deploy selection.
+
+The workload, data mounts, and ports are in `k8s/xui.yaml`; panel routing and TLS
+are in the ingress and certificate files. Keep panel access and proxy protocol
+ports separate when changing the configuration.
+
+Back up the application's database and keys before upgrades. Check the actual
+host and volume paths before moving the workload to another node.
+
+```sh
+kubectl get pods,svc,pvc -n xui
+kubectl get events -n xui --sort-by=.metadata.creationTimestamp
+```
+
+See the [repository README](../../README.md) for deployment selection.
Author	SHA1	Message	Date
forust	d85bdf5dbd	docs: sync service guides with current main ci / Compose (pull_request) Successful in 27s Details ci / Workflows (pull_request) Successful in 14s Details ci / Shell (pull_request) Successful in 34s Details ci / Python and tests (pull_request) Successful in 19s Details ci / YAML (pull_request) Successful in 17s Details ci / Dockerfiles (pull_request) Successful in 6s Details ci / Formatting (pull_request) Successful in 36s Details ci / Kubernetes (pull_request) Successful in 14s Details ci / image-plan (pull_request) Skipped Details ci / Image (${{ matrix.name }}) (pull_request) Skipped Details ci / build (pull_request) Skipped Details renovate-ci / validate-renovate (pull_request_target) Successful in 3m13s Details	2026-10-08 21:23:46 +02:00
forust	3ea181e966	Merge pull request 'chore(deps): update renovate/renovate docker tag to v44.147.0' (#114 ) from renovate/renovate-self-update into main renovate-ci / validate-renovate (push) Successful in 3m10s Details ci / Compose (push) Successful in 15s Details ci / Workflows (push) Successful in 8s Details ci / Shell (push) Successful in 21s Details ci / Python and tests (push) Successful in 10s Details ci / Formatting (push) Successful in 21s Details ci / YAML (push) Successful in 18s Details ci / Dockerfiles (push) Successful in 10s Details ci / Kubernetes (push) Successful in 14s Details ci / image-plan (push) Successful in 24s Details ci / Image (error-pages) (push) Successful in 24s Details ci / Image (forust-homepage) (push) Successful in 26s Details ci / Image (xdfnx-homepage) (push) Successful in 25s Details ci / build (push) Successful in 28s Details Reviewed-on: #114	2026-10-08 19:15:52 +00:00
renovate-bot Bot	eb2f6f7d5d	chore(deps): update renovate/renovate docker tag to v44.147.0	2026-10-08 19:15:52 +00:00
forust	67d08fc33e	Merge pull request 'chore(deps): update helm release kube-prometheus-stack to v86.3.2' (#107 ) from renovate/helm-kube-prometheus-stack into main ci / Compose (push) Canceled after 0s Details ci / Workflows (push) Canceled after 0s Details ci / Shell (push) Canceled after 0s Details ci / Python and tests (push) Canceled after 0s Details ci / Formatting (push) Canceled after 0s Details ci / YAML (push) Canceled after 0s Details ci / Dockerfiles (push) Canceled after 0s Details ci / Kubernetes (push) Canceled after 0s Details ci / image-plan (push) Canceled after 0s Details ci / Image (${{ matrix.name }}) (push) Canceled after 0s Details ci / build (push) Canceled after 0s Details Reviewed-on: #107	2026-10-08 19:15:42 +00:00
renovate-bot Bot	f748a3c7aa	chore(deps): update helm release kube-prometheus-stack to v86.3.2	2026-10-08 19:15:42 +00:00
forust	8c8ff47241	Merge pull request 'chore(deps): update helm release reloader to v2.2.18' (#102 ) from renovate/helm-reloader into main ci / Compose (push) Canceled after 0s Details ci / Workflows (push) Canceled after 0s Details ci / Shell (push) Canceled after 0s Details ci / Formatting (push) Canceled after 0s Details ci / Python and tests (push) Canceled after 0s Details ci / YAML (push) Canceled after 0s Details ci / Dockerfiles (push) Canceled after 0s Details ci / Kubernetes (push) Canceled after 0s Details ci / image-plan (push) Canceled after 0s Details ci / Image (${{ matrix.name }}) (push) Canceled after 0s Details ci / build (push) Canceled after 0s Details Reviewed-on: #102	2026-10-08 19:14:10 +00:00
renovate-bot Bot	ed2ba44bee	chore(deps): update helm release reloader to v2.2.18	2026-10-08 19:14:10 +00:00
forust	bce653ebaf	Merge pull request 'chore(deps): update all patch updates' (#101 ) from renovate/all-patch into main ci / Formatting (push) Canceled after 0s Details ci / Python and tests (push) Canceled after 0s Details ci / YAML (push) Canceled after 0s Details ci / Dockerfiles (push) Canceled after 0s Details ci / Kubernetes (push) Canceled after 0s Details ci / image-plan (push) Canceled after 0s Details ci / Image (${{ matrix.name }}) (push) Canceled after 0s Details ci / build (push) Canceled after 0s Details renovate-ci / validate-renovate (push) Successful in 2m53s Details ci / Compose (push) Canceled after 0s Details ci / Workflows (push) Canceled after 0s Details ci / Shell (push) Canceled after 0s Details Reviewed-on: #101	2026-10-08 19:13:58 +00:00
renovate-bot Bot	624ae84da1	chore(deps): update all patch updates	2026-10-08 19:13:58 +00:00
forust	f00c044f3d	Merge pull request 'fix(ci): keep build and test reports accurate' (#118 ) from fix/ci-test-output-isolation into main ci / Formatting (push) Successful in 21s Details ci / Python and tests (push) Successful in 10s Details ci / YAML (push) Successful in 8s Details ci / Dockerfiles (push) Successful in 5s Details ci / Kubernetes (push) Successful in 6s Details ci / image-plan (push) Successful in 14s Details ci / Image (error-pages) (push) Successful in 23s Details ci / Image (forust-homepage) (push) Successful in 18s Details ci / Image (xdfnx-homepage) (push) Successful in 19s Details ci / build (push) Successful in 18s Details ci / Compose (push) Successful in 13s Details ci / Workflows (push) Successful in 8s Details ci / Shell (push) Successful in 16s Details Reviewed-on: #118	2026-10-08 19:13:21 +00:00
forust	0e3035ed74	fix(ci): validate image digests and isolate test outputs ci / Compose (pull_request) Successful in 12s Details ci / Workflows (pull_request) Successful in 9s Details ci / Shell (pull_request) Successful in 21s Details ci / Formatting (pull_request) Successful in 22s Details ci / Python and tests (pull_request) Successful in 10s Details ci / YAML (pull_request) Successful in 11s Details ci / Dockerfiles (pull_request) Successful in 6s Details ci / Kubernetes (pull_request) Successful in 8s Details ci / image-plan (pull_request) Skipped Details ci / Image (${{ matrix.name }}) (pull_request) Skipped Details ci / build (pull_request) Skipped Details	2026-10-08 20:46:46 +02:00
forust	1d8eda6e5e	test: isolate CI summary and output files ci / Formatting (pull_request) Successful in 24s Details ci / Compose (pull_request) Successful in 16s Details ci / Workflows (pull_request) Successful in 9s Details ci / Shell (pull_request) Successful in 18s Details ci / Python and tests (pull_request) Successful in 11s Details ci / YAML (pull_request) Successful in 11s Details ci / Dockerfiles (pull_request) Successful in 7s Details ci / Kubernetes (pull_request) Successful in 8s Details ci / image-plan (pull_request) Skipped Details ci / Image (${{ matrix.name }}) (pull_request) Skipped Details ci / build (pull_request) Skipped Details	2026-10-08 20:20:44 +02:00
forust	fade5439c7	Merge pull request 'fix(deploy): correct service selection and recovery validation' (#117 ) from fix/cicd-review-recovery into main ci / Compose (push) Successful in 15s Details ci / Workflows (push) Successful in 7s Details ci / Shell (push) Successful in 18s Details ci / Formatting (push) Successful in 17s Details ci / Python and tests (push) Successful in 8s Details ci / Kubernetes (push) Successful in 7s Details ci / YAML (push) Successful in 11s Details ci / Dockerfiles (push) Successful in 6s Details ci / image-plan (push) Successful in 12s Details ci / Image (error-pages) (push) Successful in 16s Details ci / Image (forust-homepage) (push) Successful in 32s Details ci / Image (xdfnx-homepage) (push) Successful in 16s Details ci / build (push) Successful in 26s Details Reviewed-on: #117	2026-10-08 18:13:41 +00:00
forust	5c8bc15e60	docs(reloader): describe workload opt-in and reload policy renovate-ci / validate-renovate (push) Skipped ci / lint-compose (push) Successful in 10s Details ci / lint-actionlint (push) Successful in 7s Details ci / lint-shellcheck (push) Successful in 9s Details ci / lint-prettier (push) Successful in 19s Details ci / lint-ruff (push) Successful in 7s Details ci / lint-yaml (push) Successful in 10s Details ci / lint-dockerfiles (push) Successful in 6s Details ci / validate (push) Successful in 6s Details ci / build (push) Skipped Details ci / lint-compose (pull_request) Successful in 10s Details ci / lint-actionlint (pull_request) Successful in 5s Details ci / lint-shellcheck (pull_request) Successful in 8s Details ci / lint-prettier (pull_request) Successful in 16s Details ci / lint-ruff (pull_request) Successful in 7s Details ci / lint-yaml (pull_request) Successful in 10s Details ci / lint-dockerfiles (pull_request) Successful in 7s Details ci / validate (pull_request) Successful in 7s Details ci / build (pull_request) Skipped Details renovate-ci / validate-renovate (pull_request) Successful in 9s Details	2026-10-06 16:14:04 +02:00
forust	3c4732ae20	docs: document homelab services, deployment, and repository review	2026-10-06 16:09:50 +02:00