RollingUpdate with default maxSurge needs a spare pod the single node does not have (99% CPU requested), so multi-workload restarts end Pending and verify times out. Recreate on all replicas:1 Deployments (immich-server and bentopdf keep RollingUpdate at replicas 2). Also trims CPU/memory requests toward measured use (adguard, authentik, gitea, netbox, uptime-kuma, netbird-server) and gives traefik requests/limits so it is no longer BestEffort.
Portainer had been running for 111 days with a 512Mi request and a 2Gi limit
against 50M of measured use, on a node that is short of memory. It is a UI over
the Docker socket; nothing in the repo or the cluster depends on it.
The marker goes, not the manifests. `portainer/k8s/active` is what puts these
files in the deploy's manifest set, so without it the next push leaves the
namespace alone and the manifests stay on disk for a one-command return. This
also matters for the smoke stage: that host list is built from the active
directories, so `portainer.forust.xyz` leaves it and the new router check does
not go looking for a route to a service we just retired.
In the cluster the Deployment, the Service and both IngressRoutes are deleted.
The routes go first: leaving an IngressRoute behind a deleted Service keeps a
Traefik router pointing at nothing, which answers 502 while looking perfectly
healthy to the stage that just started checking for routers.
Deliberately kept, so this is reversible rather than destructive: the namespace,
the 2Gi `portainer-data-pvc` and both Certificates stay. Re-enabling is
`git checkout HEAD~1 -- portainer/k8s/active` plus an apply, and no Let's Encrypt
quota is spent reissuing the production certificate.
`glance` still links to `portainer.forust.xyz` and that tile will now be a 404.
Left alone on purpose. The Cloudflare record is manual and cfddns only ever
creates records, so `portainer.forust.xyz` keeps resolving until it is removed
in the dashboard, same as `dockmon.forust.xyz`.
Verified: no Traefik router matches portainer any more, all 19 hosts left in the
smoke list still have a router, and 16/16 local gates pass.
All prod IngressRoutes switch tls.certResolver to tls.secretName
backed by per-router Certificates (HTTP-01, letsencrypt-prod).
adguard-prod reuses the shared adguard-certs secret (also feeds
DoT :853); sync CronJob removed as redundant.
Traefik certificatesResolvers removed: its internal
acme-http@internal router hijacks HTTP-01 for every host while
enabled, blocking external solvers. Dormant files (kener,
downtify) converted for consistency but not applied; n8n
untouched per live-only rule.
Protect public Traefik routes with CrowdSec HTTP decisions and restore access logging for web traffic analysis.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
services marked k8s/active are applied via kubectl; the rest via docker
compose. inactive services with k8s/ keep only routing manifests
(external Services, EndpointSlices, Ingresses) to reach docker backends.
headscale/nextcloud routing moved to k8s/routing/.
validations: compose config --quiet + kubectl apply --dry-run=client.
namespace manifests applied first. pull_policy:build stacks get
build+push before up so the registry image stays fresh.
forust
renovate-bot
