100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
forust	fb400eea6e	Use plain text for the deploy request details	2026-10-06 23:44:24 +02:00
forust	0c76426c17	Show failure details in CI and deploy summaries	2026-10-06 23:44:00 +02:00
forust	d74822cd27	Add CI and deploy summaries	2026-10-06 23:28:56 +02:00
forust	b677d553b4	refactor(ci): split checks into visible jobs	2026-10-06 23:22:15 +02:00
forust	898463b759	Merge pull request 'refactor(ci): native VPS runner and durable incremental deploys' (#98) from codex/cicd-runner-deploy into main ... Reviewed-on: #98	2026-10-06 21:19:55 +00:00
forust	9a76529be8	refactor(ci): use native runner and durable incremental deploys	2026-10-06 23:08:47 +02:00
forust	5f9354b9a8	fix(edu): deploy reviewed application release with Redis authentication	2026-10-06 22:56:02 +02:00
forust	fcd16f6128	Merge pull request 'chore(deps): update renovate/renovate docker tag to v44.140.0' (#96) from renovate/renovate-self-update into main ... Reviewed-on: #96	2026-10-06 17:51:40 +00:00
forust	3c7e358dd5	Merge pull request 'chore(deps): update lscr.io/linuxserver/qbittorrent docker tag to v20' (#97) from renovate/lscr.io-linuxserver-qbittorrent-20.x into main ... Reviewed-on: #97	2026-10-06 17:51:25 +00:00
forust	7fdeacffb8	feat(monitoring): stand down Prometheus server during VM trial ... vmagent scrapes and remote-writes to VictoriaMetrics, so the Prometheus server scales to 0. Encoded as prometheusSpec.replicas in values instead of a kubectl patch, so helm keeps owning spec.replicas and Helm 4 server-side apply stops conflicting with the kubectl-patch field manager.	2026-10-06 19:29:30 +02:00
forust	cef499de73	fix(deploy): skip VMAgent in secrets check before CRD install ... check_referenced_secrets ran kubectl create on vmagent.yaml even when the VMAgent CRD is not installed yet, failing validate with 'no matches for kind VMAgent'. Apply the same skip_uninstalled_vmagent_crd guard used by both dry-run loops.	2026-10-06 19:19:47 +02:00
forust	1b70a55300	fix(ci): handle malformed push before SHA	2026-10-06 18:21:19 +02:00
forust	3f2b4e9acf	fix(deploy): skip VMAgent preflight before CRD install	2026-10-06 18:18:25 +02:00
forust	6057734a4f	fix(renovate): sync generated configmap	2026-10-06 18:08:17 +02:00
forust	8eedf8b74b	Merge pull request 'feat(monitoring): add VictoriaMetrics trial stack' (#95) from feat/victoria-metrics-migration into main ... Reviewed-on: #95	2026-10-06 16:05:46 +00:00
forust	8c0e36a5c0	feat(monitoring): replace scrape dump with vmagent	2026-10-06 18:03:59 +02:00
forust	78cd15f12c	chore(monitoring): remove vmctl backfill job ... One-shot Prometheus history backfill is complete; drop the Job and clean up related comments.	2026-10-06 17:53:24 +02:00
forust	18c633c242	feat(monitoring): add VictoriaMetrics trial stack	2026-10-06 17:45:44 +02:00
forust	4510394531	Merge pull request 'chore(deps): update renovate/renovate docker tag to v44.139.0' (#81) from renovate/renovate-self-update into main ... Reviewed-on: #81	2026-10-06 15:33:45 +00:00
forust	8ce0b809c0	Merge pull request 'chore(deps): update all minor updates' (#77) from renovate/all-minor into main ... Reviewed-on: #77	2026-10-06 15:33:28 +00:00
forust	bdcbb3d5af	Merge pull request 'chore(deps): update all patch updates' (#82) from renovate/all-patch into main ... Reviewed-on: #82	2026-10-06 15:33:09 +00:00
forust	695da1308c	Merge pull request 'fix(renovate): restore custom extraction and update groups' (#93) from fix/renovate-extraction-groups into main ... Reviewed-on: #93	2026-10-06 15:31:25 +00:00
forust	552b22cb66	fix(renovate): include self-update Compose filename	2026-10-06 15:31:25 +00:00
forust	3756e60c95	fix(renovate): restore custom extraction and update groups	2026-10-06 15:31:25 +00:00
forust	d0d217ddf4	Merge pull request 'fix(deploy): skip verify and smoke when deployment is disabled' (#94) from fix/deploy-skip-when-disabled into main ... Reviewed-on: #94	2026-10-06 15:24:17 +00:00
forust	24f84bab2f	fix(deploy): skip verification when deployment is disabled	2026-10-06 15:24:06 +00:00
forust	0b8a3c16a7	Merge pull request 'fix(glance): mount CSS from the correct ConfigMap' (#87) from fix/glance-assets into main ... Reviewed-on: #87	2026-10-06 15:17:47 +00:00
forust	e9a68aae77	fix(glance): mount CSS from the assets ConfigMap	2026-10-06 15:17:47 +00:00
forust	5359df5ed6	Merge pull request 'fix(postgres): include the required NetBox database password' (#88) from fix/postgres-env-example into main ... Reviewed-on: #88	2026-10-06 15:17:19 +00:00
forust	8aea0f0d13	fix(postgres): include required NetBox password in Compose env example	2026-10-06 15:17:19 +00:00
forust	65d4f2d482	Merge pull request 'fix(traefik): correct AdGuard and SearXNG Compose rules' (#90) from fix/compose-router-rules into main ... Reviewed-on: #90	2026-10-06 15:10:49 +00:00
forust	2fe9b7632f	fix(traefik): correct AdGuard and SearXNG Compose router expressions	2026-10-06 15:10:49 +00:00
forust	e436d89eef	Merge pull request 'fix(netbird): restore Compose setup and runtime renderer' (#86) from fix/netbird-compose-runtime into main ... Reviewed-on: #86	2026-10-06 15:10:12 +00:00
forust	8729cb5062	fix(netbird): restore Compose setup and server entrypoint	2026-10-06 15:08:46 +00:00
forust	f1e4a1088d	Merge pull request 'fix(ci): deduplicate PR checks and filter deploy triggers' (#92) from fix/ci-trigger-dedup into main ... Reviewed-on: #92	2026-10-06 15:06:47 +00:00
forust	b357ef95d8	fix(deploy): only create automatic runs for main CI	2026-10-06 17:00:05 +02:00
forust	67422663b7	fix(ci): avoid duplicate branch and PR runs	2026-10-06 16:59:26 +02:00
forust	88705fec88	Merge pull request 'fix(deploy): reject unsafe per-file pruning' (#85) from fix/deploy-prune-guard into main ... Reviewed-on: #85	2026-10-06 14:57:22 +00:00
forust	c098807aa4	fix(deploy): reject destructive per-file pruning before apply	2026-10-06 14:57:11 +00:00
forust	e1eee2d3c7	Merge pull request 'feat(reloader): enable deploy and integrate configuration reloads' (#91) from fix/reloader-integration into main ... Reviewed-on: #91	2026-10-06 14:55:16 +00:00
forust	ff83daed1e	feat(reloader): enable deployment and reload runtime-config consumers	2026-10-06 14:54:45 +00:00
forust	080ae343e6	Merge pull request 'fix(deploy): validate resolved Compose config and namespaced Secrets' (#84) from fix/deploy-validation into main ... Reviewed-on: #84	2026-10-06 14:54:27 +00:00
forust	8d3185f8ab	fix(ci): install jq for deploy validation regressions	2026-10-06 16:31:48 +02:00
forust	ff40a71145	fix(deploy): resolve Compose config and check namespaced pod Secrets	2026-10-06 15:58:44 +02:00
forust	cc9c3dea88	feat(traefik): expose insecure API on 8080 for Homarr integration ... ClusterIP access to api@internal from Homarr pod. LAN-reachable on 192.168.80.2:8080, accepted.	2026-10-06 11:27:38 +02:00
forust	815cd85b9a	feat(homarr): deploy dashboard to k8s on home subdomain ... Local-only IngressRoute (home.workstation.internal, home.gigaforust.internal), prod commented out. Compose stack for test stand.	2026-10-06 11:27:35 +02:00
forust	9021eddbc3	chore(deps): pin streaming images, isolate python and floating tags ... Python Y-bumps arrived as minor 3.11->3.14 and floating tags (:latest/:beta) were automerged blindly. Pin the linuxserver stack to digest-verified tags and keep python plus rolling images in manual review groups.	2026-10-05 23:53:38 +02:00
forust	2adf17c307	Merge pull request 'chore(deps): update all patch updates' (#76) from renovate/all-patch into main ... Reviewed-on: #76	2026-10-05 21:41:00 +00:00
forust	34f10211ab	Merge pull request 'chore(deps): update renovate/renovate docker tag to v44.136.0' (#79) from renovate/renovate-self-update into main ... Reviewed-on: #79	2026-10-05 21:39:21 +00:00
forust	8799962b1c	Revert "feat(adguard): add netbird sidecar" ... This reverts commit 7a81b4ea8b.	2026-10-04 17:40:19 +02:00
forust	fb80024fa2	feat(streaming): add bazarr for subtitles	2026-10-04 17:25:00 +02:00
forust	0f1a788874	Merge pull request 'feat(streaming): compose *arr streaming stack (k8s routing)' (#78) from feat/streaming-compose-test into main ... Reviewed-on: #78	2026-10-04 14:04:28 +00:00
forust	39df442e60	fix(streaming): trailing newline for yamllint	2026-10-04 16:03:40 +02:00
forust	62a451773e	feat(streaming): compose *arr streaming stack (k8s routing)	2026-10-04 15:57:08 +02:00
forust	f196099491	feat(adguard): add DNS readiness probe	2026-10-03 22:39:25 +02:00
forust	66502b8279	fix(traefik): protect dashboard with security-chain	2026-10-03 22:39:25 +02:00
forust	9018c091fa	feat(uptime-kuma): add ServiceMonitor, alerts and secrets example	2026-10-03 22:39:24 +02:00
forust	114608af2f	fix(uptime-kuma): label service and name http port	2026-10-03 22:39:23 +02:00
forust	51a73fb213	chore(gitea): switch domain to git.forust.xyz (28.0.0 update)	2026-10-03 22:37:48 +02:00
forust	939fad4a23	chore(renovate,crowdsec): group minor/patch updates and whitelist VPS ... Renovate: group all minor updates and all patch updates into reviewable PRs. Crowdsec: whitelist static VPS IP. Remove stale cloudflared/k8s/active marker.	2026-10-03 11:57:46 +02:00
forust	12d5cc6202	Merge pull request 'chore(deps): update docker.gitea.com/gitea docker tag to v28' (#74) from renovate/docker.gitea.com-gitea-28.x into main ... Reviewed-on: #74	2026-10-03 09:44:13 +00:00
forust	a6b84d86b4	Merge pull request 'chore(deps): update gitea/gitea docker tag to v28' (#75) from renovate/gitea-gitea-28.x into main ... Reviewed-on: #75	2026-10-03 09:44:06 +00:00
forust	f3a5cedc80	Merge pull request 'chore(deps): update renovate/renovate docker tag to v44.132.2' (#69) from renovate/renovate-self-update into main ... Reviewed-on: #69	2026-10-03 09:31:07 +00:00
forust	e208ccae62	Merge pull request 'chore(deps): update ghcr.io/alexta69/metube docker tag to v2026.09.29' (#70) from renovate/container-patch-updates into main ... Reviewed-on: #70	2026-10-03 09:20:36 +00:00
forust	d3892d2ed5	Merge pull request 'chore(deps): update ghcr.io/henriquesebastiao/downtify docker tag to v3.4.0' (#71) from renovate/ghcr.io-henriquesebastiao-downtify-3.x into main ... Reviewed-on: #71 Reviewed-by: forust <vzlomdsisma@gmail.com>	2026-10-03 09:20:06 +00:00
forust	7164b48275	Merge pull request 'chore(deps): update ghcr.io/lukegus/termix docker tag to v2.9.0' (#72) from renovate/ghcr.io-lukegus-termix-2.x into main ... Reviewed-on: #72	2026-10-03 09:19:37 +00:00
forust	956596e6aa	Merge pull request 'chore(deps): update docker.n8n.io/n8nio/n8n docker tag to v2.42.2' (#68) from renovate/docker.n8n.io-n8nio-n8n-2.x into main ... Reviewed-on: #68	2026-10-03 09:17:01 +00:00
forust	65709e005b	Merge pull request 'chore(deps): update netbirdio/netbird-server docker tag to v0.80.0' (#73) from renovate/netbirdio-netbird-server-0.x into main ... Reviewed-on: #73	2026-10-03 09:16:28 +00:00
forust	78e6363fe2	chore(gitea): add git.forust.xyz route	2026-10-03 11:07:48 +02:00
forust	22c2f1e108	extract dtek_notif subtree to its own repo	2026-10-03 10:48:01 +02:00
forust	64ba4ce9f1	Revert "chore(prometheus): drop dead grafana routes and cert" ... This reverts commit 314ec0cda7.	2026-10-03 10:40:39 +02:00
forust	7a81b4ea8b	feat(adguard): add netbird sidecar	2026-10-03 10:36:15 +02:00
forust	0859479c0f	feat(ingress): replace traefik crowdsec plugin with firewall bouncer ... Move L3 enforcement to the host firewall-bouncer (systemd, nftables): drop the Traefik plugin, its secrets volume and the crowdsec Middleware, remove bouncer refs from all IngressRoutes. Disable the http-generic-bf scenario (403-burst bans hurt legit automation under L3 enforcement). Add a Gateway API PoC for homepages prod and CrowdSec PrometheusRule alerts.	2026-09-30 20:14:25 +02:00
forust	872f64b887	fix(ci): silence intentional SC2029 in ssh-run.sh	2026-09-29 14:54:17 +02:00
forust	a90fb19ed4	fix(traefik): size probes for HDD stalls ... Single replica is the whole ingress; liveness kills at 2s timeouts took every public service down in a loop. Same stall-sized budgets as postgres/metallb. Applied live via helm (pinned 41.5.0, values from git).	2026-09-29 14:47:53 +02:00
forust	6f4cd03f4b	feat(deploy): serialize apply stages and wait for calm node ... apply-k8s and apply-compose share a workstation flock so host docker churn never overlaps cluster churn. Each helm upgrade and the apply loop wait up to 10m for load <28 first, so a deploy never piles onto an already-hot node (the load-40/netbird-death/pending-helm spiral).	2026-09-29 14:42:56 +02:00
forust	e56662194b	fix(ci): log in to registry for manifest-only main pushes ... The pin step writes manifest PUTs on every main push, but login was gated on services != ''. Manifest-only pushes skipped login and pushed anonymously (401); this only worked before via a stale persistent login on the old runner.	2026-09-29 14:21:57 +02:00
forust	4856348a6e	chore(searxng,glance): disable services ... Workloads, services, routes and certs removed from the cluster. Configs, manifests and PVCs kept for easy re-enable via k8s/active.	2026-09-29 13:59:59 +02:00
forust	49d0da1dd0	chore(termix): disable service ... Deployment was stuck in ImagePullBackOff and only generated log churn. Workload, service, routes and certs removed from the cluster; PVC, config and manifests kept so it can be re-enabled by restoring k8s/active.	2026-09-29 13:58:27 +02:00
forust	6bc938436f	Merge pull request 'chore(deps): update grafana/grafana docker tag to v13.2.3' (#67) from renovate/grafana-monorepo into main ... Reviewed-on: #67	2026-09-29 11:57:17 +00:00
forust	5dcad7eb38	fix(ci): resolve uv from BIN_DIR in install-ci-tools ... Callers prepend BIN_DIR to PATH only after the script exits, so the bare uv invocation in install_uv_tool died with 127 on clean runners. Export BIN_DIR to PATH inside the script and invoke the just-installed binary by absolute path.	2026-09-29 13:31:28 +02:00
forust	d0872bc918	feat(deploy): autodeploy off unless AUTODEPLOY=true ... Pushes no longer reach the cluster by default; set the AUTODEPLOY repo variable to 'true' to re-enable, or dispatch manually. Skips propagate through the existing needs chain.	2026-09-29 12:54:17 +02:00
forust	91dd749a29	feat(deploy): AUTODEPLOY kill-switch via repo variable ... Set AUTODEPLOY=false under Settings -> Actions -> Variables and pushes stop deploying with no commit; unset means on. Manual Run workflow always bypasses the switch. The existing needs/skipped chaining propagates the skip through verify and smoke untouched.	2026-09-29 12:52:35 +02:00
forust	b6e1dc0362	fix(immich): size postgres probes for HDD stalls ... Postmaster was SIGKILLed in a loop: 70s fsync stalls on the loaded rotational disk outlasted the 5-minute startup budget and the 60s liveness tolerance, and every kill bought another full WAL replay. Startup budget 15min, liveness 5x60s. Already applied live with kubectl; this keeps git in sync.	2026-09-29 12:36:04 +02:00
forust	a2af854867	fix(alerts): measure traefik latency per route via exported_service ... The service label the rule grouped by is the scrape target name, not the backend: with honorLabels=false the per-route value traefik emits is renamed to exported_service, so the old rule measured one global aggregate and both exclusions matched nothing. Group the latency and 5xx alerts by exported_service with exclusions in traefik's real namespace-routename-hash format. Already applied live with kubectl; this commit keeps git in sync.	2026-09-29 09:55:02 +02:00
forust	314ec0cda7	chore(prometheus): drop dead grafana routes and cert ... No grafana is deployed from this stack, so grafana-prod/grafana-local only ever served 503s. Live objects already deleted directly; this keeps git from resurrecting them on the next deploy.	2026-09-29 09:33:38 +02:00
forust	e45ae10204	feat(immich): migrate postgres 14 to 16 with dump/restore ... Reverts the v16 revert: data migrated via pg_dumpall from PG14 into a fresh PG16 data directory (same extension versions vectorchord 0.4.3/pgvectors 0.2.0). Verified 1944 assets, 2 users, 10 albums on 16.10. Note: the restore OOMed once at the 2Gi limit during index builds and recovered via WAL replay; worth watching under PG16 load.	2026-09-29 09:16:58 +02:00
forust	357ee26111	Revert "Merge pull request 'chore(deps): update ghcr.io/immich-app/postgres docker tag to v16' (#65) from renovate/ghcr.io-immich-app-postgres-16.x into main" ... This reverts commit 0c9743e241, reversing changes made to 2cb06debc5.	2026-09-29 08:25:43 +02:00
forust	436fd1ecae	chore: extract userbot subtree to its own repo ... userbot/ (bot plus panel) now lives at /home/forust/userbot as a clone of forust/userbot instead of a subtree in homelab. Cleans up the pipeline references that only existed for it: scan-deps, test-backend and test-frontend jobs, the userbot build matrix entries, the deploy panel hook, and the userbot-only pyrightconfig. Running cluster workloads are untouched; homelab just stops building and testing upstream's code.	2026-09-29 08:24:14 +02:00
forust	a564dd8b67	fix(alerts): exclude netbird streams from traefik latency alert ... SignalExchange ConnectStream holds 60s gRPC streams by design; at night they exceed 5% of samples and pin P95 to the 5.0s bucket ceiling, flapping the alert. Also fixes the stale xui exclusion pattern, which matched no real service label.	2026-09-29 08:12:07 +02:00
forust	c2c90c490d	Merge pull request 'chore(deps): update ghcr.io/autobrr/netronome docker tag to v0.15.0' (#62) from renovate/ghcr.io-autobrr-netronome-0.x into main ... Reviewed-on: #62	2026-09-28 21:05:42 +00:00
forust	3ecc12300a	Merge pull request 'chore(deps): update ghcr.io/alexta69/metube docker tag to v2026.09.28' (#61) from renovate/container-patch-updates into main ... Reviewed-on: #61	2026-09-28 21:05:33 +00:00
forust	2a5d690e34	Merge pull request 'chore(deps): update netbirdio/dashboard docker tag to v2.94.0' (#63) from renovate/netbirdio-dashboard-2.x into main ... Reviewed-on: #63	2026-09-28 21:05:25 +00:00
forust	a8c4e9bcbe	Merge pull request 'chore(deps): update renovate/renovate docker tag to v44.117.0' (#64) from renovate/renovate-self-update into main ... Reviewed-on: #64	2026-09-28 21:02:09 +00:00
forust	0c9743e241	Merge pull request 'chore(deps): update ghcr.io/immich-app/postgres docker tag to v16' (#65) from renovate/ghcr.io-immich-app-postgres-16.x into main ... Reviewed-on: #65	2026-09-28 21:01:58 +00:00
forust	2cb06debc5	fix(deploy): retry registry lookups with a timeout ... A single blink of the registry failed render_pinned for the whole file and redded the apply stage. registry_digest now retries 3 times under a 25s timeout with a warning per attempt; empty still means unresolvable and callers report it by name as before.	2026-09-28 22:52:54 +02:00
forust	a6af69dca0	fix(k8s): Recreate singletons and trim requests for scheduler headroom ... RollingUpdate with default maxSurge needs a spare pod the single node does not have (99% CPU requested), so multi-workload restarts end Pending and verify times out. Recreate on all replicas:1 Deployments (immich-server and bentopdf keep RollingUpdate at replicas 2). Also trims CPU/memory requests toward measured use (adguard, authentik, gitea, netbox, uptime-kuma, netbird-server) and gives traefik requests/limits so it is no longer BestEffort.	2026-09-28 22:36:46 +02:00
forust	76f39da90c	fix(ci): skip heavy jobs on renovate branches, automerge digest and patch ... Renovate branches only carry version/digest bumps, so scan-deps, test-backend, test-frontend and build just burn runner time on the box that also serves prod. Static checks and validate still run. Digest and patch updates automerge (playwright, helm and major rules below still override to no-automerge). Also replaces deprecated helm --atomic with --wait --rollback-on-failure.	2026-09-28 22:19:57 +02:00
forust	86730ff0c4	Merge pull request 'chore(deps): update ghcr.io/c4illin/convertx docker tag to v0.19.0' (#55) from renovate/ghcr.io-c4illin-convertx-0.x into main ... Reviewed-on: #55	2026-09-28 19:57:17 +00:00
forust	deeaefa695	Merge pull request 'chore(deps): update ghcr.io/henriquesebastiao/downtify docker tag to v3.2.0' (#60) from renovate/ghcr.io-henriquesebastiao-downtify-3.x into main ... Reviewed-on: #60	2026-09-28 19:57:05 +00:00