docs: sync service guides with current main
ci / Compose (pull_request) Successful in 27s
ci / Workflows (pull_request) Successful in 14s
ci / Shell (pull_request) Successful in 34s
ci / Python and tests (pull_request) Successful in 19s
ci / YAML (pull_request) Successful in 17s
ci / Dockerfiles (pull_request) Successful in 6s
ci / Formatting (pull_request) Successful in 36s
ci / Kubernetes (pull_request) Successful in 14s
ci / image-plan (pull_request) Skipped
ci / Image (${{ matrix.name }}) (pull_request) Skipped
ci / build (pull_request) Skipped
renovate-ci / validate-renovate (pull_request_target) Successful in 3m13s

This commit is contained in:
forust committed 2026-10-08 21:23:46 +02:00
commit d85bdf5dbd
126 files changed
+4971 -4076

No files matched your search

+59 -76
View File
@@ -1,39 +1,40 @@
# Repository review
# Repository review (6 October 2026 baseline)
Reviewed the tracked tree at `cc9c3de` and read the live workstation state on
6 October 2026. Changes are split into documentation and individual fix branches,
all based on that main commit. The original local checkout and its uncommitted
monitoring changes were preserved. No deployment was performed.
This records the tracked tree at `cc9c3de` and the workstation state observed on
6 October 2026. It is a historical review, not a current runtime inventory. The
listed code fixes have since merged into `main`; EDU ownership has moved to the
separate repository described in [the handoff record](../.gitea/EDU_HANDOFF.md).
See the [CI and deployment guide](../.gitea/README.md) and
[runner and recovery guide](../.gitea/runner/README.md) for the current workflow.
No deployment was performed during the original review.
## Confirmed problems with prepared fixes
## Findings at the baseline and current status
| Priority | Problem and consequence | Fix branch |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- |
| High | `APPLY_PRUNE=true` is passed to each individual manifest apply. Each invocation sees only that file's desired objects and can delete other resources selected by the shared label. | `fix/deploy-prune-guard` |
| High | Deploy validates Compose with interpolation and env/path resolution disabled. Required settings can pass validation and then fail during apply after other workloads have changed. | `fix/deploy-validation` |
| Medium | Secret validation is text-based and compares names across all namespaces. A Secret elsewhere can hide a missing local Secret; mounted Secrets are also missed. | `fix/deploy-validation` |
| Medium | Compose CI misses `postgres/shared-compose.yaml`, `netbird/client.compose.yaml`, and `renovate/renovate-compose.yaml`. | `fix/deploy-validation` |
| Medium | NetBird Compose mounts `entrypoint.sh`, but it is absent. Its README also calls a missing `setup.sh`; a fresh checkout cannot start this stack as documented. | `fix/netbird-compose-runtime` |
| Medium | Glance's CSS mount uses `glance-config`, whose keys do not include `user.css`. That key is in `glance-assets`; the pod's subPath mount cannot be prepared correctly. | `fix/glance-assets` |
| Medium | The shared PostgreSQL initializer requires `NETBOX_DB_PASSWORD`, but the Compose env example omits it. Following the example leaves first initialization incomplete. | `fix/postgres-env-example` |
| Medium | EDU's Compose env example uses old credential names and full URL variables, while the code reads `KEEPER_*` and paths under `EDU_URL_BASE`. | `fix/session-keeper-reliability` |
| Medium | Session keeper HTTP calls have no timeouts. Its Redis cookie never expires, probes only check existence, and its logs include cookies. A hung or failed refresh can leave a stale session appearing ready. | `fix/session-keeper-reliability` |
| Medium | AdGuard's DoH and SearXNG's Compose rules put Boolean expressions inside `Host(...)`. They are invalid router expressions despite valid YAML. | `fix/compose-router-rules` |
| Priority | Finding at the baseline | Current status |
| -------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| High | Per-file `APPLY_PRUNE=true` could delete resources selected by a shared label. | The deploy workflow rejects unsafe pruning before applying resources. |
| High | Compose validation did not resolve the local configuration required at deploy time. | Preflight resolves the selected Compose configuration before apply. |
| Medium | Secret validation could miss namespace-specific and mounted Secret references. | Preflight checks rendered references in their namespaces, including mounted and projected Secrets. |
| Medium | Compose CI missed manual entry points such as `shared-compose.yaml` and `client.compose.yaml`. | CI checks all tracked Compose files. |
| Medium | NetBird Compose referenced missing setup and renderer files. | The setup and renderer files are now present; Compose remains a manual alternative to the active Kubernetes deployment. |
| Medium | Glance mounted its CSS from the wrong ConfigMap. | The mount now uses the ConfigMap that contains `user.css`. |
| Medium | The PostgreSQL env example omitted the required NetBox password. | The example now includes the required variable. |
| Medium | The former EDU code had stale Compose variable names and session reliability problems. | EDU workloads and their fixes moved out of this repository; see the handoff record. |
| Medium | AdGuard DoH and SearXNG Compose router expressions used invalid `Host(...)` syntax. | The router expressions now follow Traefik's rule syntax. |
Traefik matchers should be combined as `Host(a) || Host(b)`; the rule syntax is
described in the [Traefik rules documentation](https://doc.traefik.io/traefik/reference/routing-configuration/http/routing/rules-and-priority/).
The fix retains the DoH path constraint for both hostnames.
The prune fix deliberately rejects the unsafe option. It does not introduce
automatic deletion under a different implementation. Prune defaults to false,
and no tracked resource currently carries the selector label, so this is a
latent defect rather than evidence of a live deletion incident.
The current deploy workflow deliberately rejects the unsafe prune option. It
does not introduce automatic deletion under a different implementation. The
baseline finding was a configuration risk, not evidence of a live deletion
incident.
The session fix bounds HTTP and Redis calls, validates required credentials,
sets a cookie lifetime of two refresh intervals, and marks success only after
publishing the verified cookie. With the default ten-minute interval, an outage
longer than twenty minutes will make the existing Redis-key readiness checks fail.
That is an intentional change from indefinite apparent readiness.
The former session fix bounded HTTP and Redis calls, validated credentials, set
a cookie lifetime of two refresh intervals, and marked success only after
publishing the verified cookie. The service is now owned by the EDU repository;
see that repository for its current implementation.
The deployment fix extracts required pod Secret references from rendered JSON,
checks their namespaces, includes init containers, image-pull credentials, and
@@ -61,51 +62,34 @@ reviewed local commit. It has untracked host configuration and a separate
| Default `local-path` has reclaim policy Delete, while many existing PVs have been changed to Retain. | Current retention is partly live state. Recreating a claim can get a different policy from the old PV. |
| NetBird, NetBox media/reports/scripts, EDU Redis, Homarr, and VictoriaMetrics have Delete-policy PVs. | Deleting their claims can delete important state. Plan backup and retention changes before namespace cleanup. |
The monitoring files already modified in the user's local tree correspond to the
live migration. They are excluded from these branches. Reconcile that work before
using this review's baseline to deploy monitoring.
The VictoriaMetrics monitoring trial later merged into `main` in PR #95. The
first row above records the state before that change. Read
[`prometheus-stack/README.md`](../prometheus-stack/README.md) for the current
tracked monitoring configuration; the live observations in this section remain
a snapshot from 6 October.
## Remaining work
## Current recovery limits
These need recovery design or infrastructure decisions rather than a small
configuration correction:
The deployment controller and its recovery process changed after this review.
The current operator workflow is documented in the
[runner and recovery guide](../.gitea/runner/README.md). The remaining boundaries
are:
- **SSH apply retries can replace the rollback baseline.** `ssh-run.sh` retries
exit 255, including `apply-k8s`; every new invocation publishes a fresh snapshot.
If the first attempt already changed workloads, the retry snapshots that partial
state. Preserve a run-specific original baseline and verify it across retries.
- **Rollback can exceed the job budget.** Verification is parallel, but
`rollback_workloads` is serial with a five-minute limit per workload. The
thirty-minute job budget can expire before recovery finishes. Bound recovery
concurrency and account for both phases before choosing a new timeout.
- **Snapshot collection is allowed to fail.** Generation and workload snapshot
errors are warnings; verify can fall back to all workloads. A snapshot failure
must not permit unrelated workloads to be selected for automatic undo.
- **Rollback uses the previous revision, not the captured revision.** `rollout undo`
without an explicit revision cannot guarantee restoration to the snapshot after
retries or intervening rollouts. First deployments also have no previous revision.
- **Manual deploy dispatch bypasses the CI-success trigger.** Either validate the
target commit's successful CI run or document manual dispatch as an operator
override with its own required checks.
- **Direct Traefik API exposure is unauthenticated.** The latest local commit
explicitly added it for Homarr. Preserve that integration while choosing a
cluster-internal authenticated path or a verified network restriction; do not
simply disable an integration that is already in use.
- **Storage retention and backup are not reproducible as a whole.** Defaults and
several important PV policies are Delete. There is no repository-wide backup
schedule. Existing PVC StorageClass changes require migration rather than an
in-place YAML edit.
- **MeTube downloads are temporary on Kubernetes.** `/downloads` is a 20 GiB
emptyDir. Decide whether pod replacement should discard files or whether it
should use persistent storage. Compose uses a host directory instead.
- **First-time activation needs a bootstrap path.** Deploy validation dry-runs
namespaced resources before the apply stage creates namespaces and installs
selected charts. On a fresh cluster, missing namespaces and CRDs need separate
preparation; activation is not a complete installer.
- Kubernetes recovery can restore captured workload revisions. It does not
restore ConfigMaps, Secrets, database schemas, or persistent data.
- Compose recovery is manual. It uses saved resolved configuration, but it does
not restore volume data or reverse database migrations.
- Removed resources require manual review and removal; the deploy workflow does
not prune them automatically.
- Plan mode does not create namespaces. During apply, server validation for new
namespaces runs after namespace creation and chart installation; a failed
check can leave an empty namespace.
- Storage policy and backup coverage remain service-specific. Check the live PV,
PVC, and backup state before changing stateful workloads.
## Validation
Baseline lint checks passed for Python, shell, workflows, YAML, standard Compose
At the review baseline, lint checks passed for Python, shell, workflows, YAML, standard Compose
files, and Kubernetes resources with available schemas. Kubeconform found 347
resources in 174 files: 201 valid, 146 skipped CRDs, zero invalid resources.
That skip count matters: passing schema validation does not validate Traefik rule
@@ -124,8 +108,8 @@ Fix validation covers:
documented Traefik grammar. They were not exercised on the live proxy.
- Prune rejection before any cluster invocation.
All seven fix branches and the documentation branch merged together without
conflicts in a disposable validation worktree. The combined tree passed the
At the time of review, all seven fix branches and the documentation branch
merged together in a disposable validation worktree. That combined tree passed the
CI-equivalent local checks, Markdown formatting/lint and link checks, all 35
Compose structure checks, and 11 Python regression tests plus the shell
validation regressions. CRD server-side validation and live rollout tests were
@@ -135,17 +119,16 @@ Runtime tests use fixtures and mocks, not production credentials. Live checks re
workload metadata, storage policies, chart versions, and container state only.
They did not read Secret contents or change services.
## Reloader follow-up
## Reloader follow-up (baseline)
`fix/reloader-integration` adds the active marker and opt-in annotations to 28
`fix/reloader-integration` added the active marker and opt-in annotations to
application Deployments/StatefulSets that consume runtime ConfigMaps or Secrets.
It corrects AdGuard's misplaced pod-template annotation. The Helm settings use
It corrected AdGuard's misplaced pod-template annotation. The Helm settings use
annotation-based reloads, keep global auto-reload disabled, and ignore Jobs and
CronJobs. PostgreSQL workloads are excluded because their credential variables
and init scripts are only effective on an empty data directory.
The controller was already running on workstation when inspected. Its live
configuration is unchanged by the branch: merge and deploy the integration to
apply the new policy and application annotations. Configuration reload behavior
was checked against the pinned chart, with Helm rendering and manifest validation;
no production configuration was changed to provoke a test restart.
The controller was running on the workstation when inspected. The original
review checked configuration against the pinned chart with Helm rendering and
manifest validation; it did not change production configuration to provoke a
test restart or confirm every application's live reload behavior.