Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 forust
|
1dec4de708 |
No files matched your search
@@ -1,191 +0,0 @@
|
|||||||
# Инструкция: Анализ хранилища Kubernetes и настройка NFS
|
|
||||||
|
|
||||||
## Цель
|
|
||||||
Проанализировать текущую конфигурацию хранилища Kubernetes и подготовить план внедрения NFS StorageClass для сохранения данных при удалении namespace.
|
|
||||||
|
|
||||||
## 1. Собрать информацию о кластере
|
|
||||||
|
|
||||||
### 1.1. Версия Kubernetes и тип дистрибутива
|
|
||||||
```bash
|
|
||||||
kubectl version --short
|
|
||||||
# или
|
|
||||||
kubectl version
|
|
||||||
```
|
|
||||||
|
|
||||||
Определить, используется ли k3s, k8s, microk8s и т.д.:
|
|
||||||
```bash
|
|
||||||
# Проверить наличие k3s
|
|
||||||
which k3s
|
|
||||||
# Проверить процесс
|
|
||||||
ps aux | grep -E 'kube|k3s'
|
|
||||||
```
|
|
||||||
|
|
||||||
### 1.2. StorageClass
|
|
||||||
```bash
|
|
||||||
kubectl get storageclass -o wide
|
|
||||||
```
|
|
||||||
|
|
||||||
Запомнить:
|
|
||||||
- `PROVISIONER` — какой драйвер используется
|
|
||||||
- `RECLAIMPOLICY` — Delete или Retain
|
|
||||||
- Какой StorageClass помечен как `(default)`
|
|
||||||
|
|
||||||
### 1.3. Существующие PV и PVC
|
|
||||||
```bash
|
|
||||||
kubectl get pv -o wide
|
|
||||||
kubectl get pvc --all-namespaces
|
|
||||||
```
|
|
||||||
|
|
||||||
Посмотреть, какие PVC привязаны к каким PV, и какой reclaimPolicy у PV.
|
|
||||||
|
|
||||||
### 1.4. Нода и диски
|
|
||||||
```bash
|
|
||||||
# Список нод
|
|
||||||
kubectl get nodes -o wide
|
|
||||||
|
|
||||||
# На каждой ноде (через ssh или локально):
|
|
||||||
lsblk
|
|
||||||
df -h
|
|
||||||
cat /etc/fstab
|
|
||||||
```
|
|
||||||
|
|
||||||
Определить:
|
|
||||||
- Есть ли отдельный раздел/диск для данных
|
|
||||||
- Куда смонтированы разделы
|
|
||||||
- Сколько свободного места
|
|
||||||
- Есть ли монтирование NTFS-разделов (как `/media/forust/Programs`)
|
|
||||||
|
|
||||||
### 1.5. Где local-path хранит данные (для k3s)
|
|
||||||
```bash
|
|
||||||
ls -la /var/lib/rancher/k3s/storage/ 2>/dev/null
|
|
||||||
# или для microk8s
|
|
||||||
ls -la /var/snap/microk8s/common/ 2>/dev/null
|
|
||||||
```
|
|
||||||
|
|
||||||
## 2. Анализ: сохраняются ли данные при удалении namespace?
|
|
||||||
|
|
||||||
| Сценарий | Результат |
|
|
||||||
|---|---|
|
|
||||||
| `kubectl delete ns <ns>` | Все PVC в namespace удаляются |
|
|
||||||
| PVC → PV c `reclaimPolicy: Delete` | PV и данные удалены |
|
|
||||||
| PVC → PV c `reclaimPolicy: Retain` | PV остаётся (статус Released), данные целы |
|
|
||||||
|
|
||||||
**Вывод:** Если reclaimPolicy в StorageClass = `Delete`, то данные **пропадут**. Если `Retain` — сохранятся.
|
|
||||||
|
|
||||||
## 3. План внедрения NFS
|
|
||||||
|
|
||||||
### 3.1. Проверить, установлен ли NFS
|
|
||||||
```bash
|
|
||||||
which nfsstat exportfs mount.nfs
|
|
||||||
systemctl status nfs-server 2>/dev/null || systemctl status nfs-kernel-server 2>/dev/null
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.2. Выбрать директорию для NFS-экспорта
|
|
||||||
|
|
||||||
Варианты (выбрать подходящий):
|
|
||||||
- `/var/lib/k8s-nfs/` — на корневом разделе
|
|
||||||
- `<путь к отдельному разделу>/k8s-nfs/` — если есть отдельный диск/раздел
|
|
||||||
- Не рекомендуется использовать NTFS-раздел (проблемы с правами и производительностью)
|
|
||||||
|
|
||||||
Требования:
|
|
||||||
- Файловая система: ext4 или xfs (не ntfs!)
|
|
||||||
- Достаточно свободного места
|
|
||||||
- Права: `755`, владелец root
|
|
||||||
|
|
||||||
### 3.3. Установить NFS-сервер
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Debian/Ubuntu
|
|
||||||
apt update && apt install -y nfs-kernel-server
|
|
||||||
|
|
||||||
# RHEL/Fedora
|
|
||||||
dnf install -y nfs-utils
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.4. Настроить экспорт
|
|
||||||
|
|
||||||
Создать директорию:
|
|
||||||
```bash
|
|
||||||
mkdir -p /var/lib/k8s-nfs
|
|
||||||
chmod 755 /var/lib/k8s-nfs
|
|
||||||
```
|
|
||||||
|
|
||||||
Добавить в `/etc/exports`:
|
|
||||||
```
|
|
||||||
/var/lib/k8s-nfs *(rw,sync,no_subtree_check,no_root_squash)
|
|
||||||
```
|
|
||||||
|
|
||||||
Применить:
|
|
||||||
```bash
|
|
||||||
exportfs -rav
|
|
||||||
```
|
|
||||||
|
|
||||||
Проверить:
|
|
||||||
```bash
|
|
||||||
showmount -e localhost
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.5. Выбрать способ интеграции с Kubernetes
|
|
||||||
|
|
||||||
#### Вариант A: nfs-subdir-external-provisioner (проще)
|
|
||||||
```bash
|
|
||||||
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
|
|
||||||
helm install nfs-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
|
|
||||||
--namespace kube-system \
|
|
||||||
--set nfs.server=127.0.0.1 \
|
|
||||||
--set nfs.path=/var/lib/k8s-nfs \
|
|
||||||
--set storageClass.name=nfs \
|
|
||||||
--set storageClass.defaultClass=false \
|
|
||||||
--set storageClass.reclaimPolicy=Retain
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Вариант B: NFS CSI Driver
|
|
||||||
```bash
|
|
||||||
helm repo add csi-driver-nfs https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts
|
|
||||||
helm install csi-driver-nfs csi-driver-nfs/csi-driver-nfs --namespace kube-system
|
|
||||||
```
|
|
||||||
|
|
||||||
После установки CSI драйвера создать StorageClass:
|
|
||||||
```yaml
|
|
||||||
apiVersion: storage.k8s.io/v1
|
|
||||||
kind: StorageClass
|
|
||||||
metadata:
|
|
||||||
name: nfs
|
|
||||||
provisioner: nfs.csi.k8s.io
|
|
||||||
parameters:
|
|
||||||
server: 127.0.0.1
|
|
||||||
share: /var/lib/k8s-nfs
|
|
||||||
reclaimPolicy: Retain
|
|
||||||
volumeBindingMode: Immediate
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3.6. Проверить результат
|
|
||||||
```bash
|
|
||||||
kubectl get storageclass
|
|
||||||
kubectl get pods -n kube-system | grep -E 'nfs|provisioner'
|
|
||||||
```
|
|
||||||
|
|
||||||
## 4. Итоговая конфигурация
|
|
||||||
|
|
||||||
После внедрения в кластере будет два StorageClass:
|
|
||||||
|
|
||||||
| Имя | Provisioner | ReclaimPolicy | Назначение |
|
|
||||||
|---|---|---|---|
|
|
||||||
| `local-path` (default) | rancher.io/local-path | Delete | Временные данные, stateless |
|
|
||||||
| `nfs` | nfs-subdir-external-provisioner или nfs.csi.k8s.io | Retain | Данные, которые нужно сохранять |
|
|
||||||
|
|
||||||
**Главное преимущество:** PVC c `storageClassName: nfs` при удалении namespace сохраняют данные на диске, так как NFS-провизор использует `reclaimPolicy: Retain` или файлы физически остаются в NFS-экспорте.
|
|
||||||
|
|
||||||
## 5. Ответы на частые вопросы
|
|
||||||
|
|
||||||
**В:** Не упадёт ли local-path при установке NFS?
|
|
||||||
**О:** Нет, они независимы. local-path продолжает работать как обычно.
|
|
||||||
|
|
||||||
**В:** Данные NFS и local-path будут на одном диске?
|
|
||||||
**О:** Да, можно настроить оба на одном разделе, в разных каталогах.
|
|
||||||
|
|
||||||
**В:** Что если у меня несколько нод?
|
|
||||||
**О:** NFS сервер нужно поднять на одной ноде, а с других нод должна быть доступна шари. Для multi-node лучше использовать отдельный сервер или distributed storage (Longhorn, Rook/Ceph).
|
|
||||||
|
|
||||||
**В:** Можно ли использовать существующий NTFS-раздел для NFS?
|
|
||||||
**О:** Не рекомендуется — NTFS не поддерживает права Linux (no_root_squash не сработает корректно), возможны проблемы с блокировками и производительностью.
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
root = true
|
|
||||||
|
|
||||||
[*]
|
|
||||||
indent_style = space
|
|
||||||
indent_size = 2
|
|
||||||
end_of_line = lf
|
|
||||||
charset = utf-8
|
|
||||||
trim_trailing_whitespace = true
|
|
||||||
insert_final_newline = true
|
|
||||||
|
|
||||||
[*.{yml,yaml}]
|
|
||||||
indent_size = 2
|
|
||||||
|
|
||||||
[*.{json,jsonc}]
|
|
||||||
indent_size = 2
|
|
||||||
|
|
||||||
[*.md]
|
|
||||||
trim_trailing_whitespace = false
|
|
||||||
|
|
||||||
[*.py]
|
|
||||||
indent_size = 4
|
|
||||||
|
|
||||||
[{Makefile,makefile}]
|
|
||||||
indent_style = tab
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
# EDU ownership handoff
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
The EDU ownership handoff is complete. The homelab repository no longer owns
|
|
||||||
EDU workloads, images, routes, alerts, or deployment selection. The EDU
|
|
||||||
repository is the only deployment owner: [forust/edu-master](https://git.forust.xyz/forust/edu-master).
|
|
||||||
|
|
||||||
Homelab PRs #99 and #105 are merged. PR #105 removed the EDU subtree and its
|
|
||||||
build, deploy, rollback, verification, route-probe, and registry references.
|
|
||||||
It also added the serial image build matrix for the homelab services. This
|
|
||||||
handoff record is the only remaining EDU-specific file in homelab Git.
|
|
||||||
|
|
||||||
The dedicated workstation checkout is `/srv/edu-master`, at release
|
|
||||||
`4f2b2a0e37dc11ac2c75441a15076c178e219d37`. It contains `k8s/active`; root
|
|
||||||
`active` is absent. The old untracked `/srv/homelab/edu_master` checkout was
|
|
||||||
moved outside the homelab repository to
|
|
||||||
`/srv/edu-master-legacy-archive-20261007/edu_master`. Its private files remain
|
|
||||||
mode `0600` inside an archive directory with mode `0700`. The homelab deploy
|
|
||||||
checkout has no EDU marker or tracked EDU application/deployment files.
|
|
||||||
`AUTODEPLOY=false` remains in place for homelab deployment.
|
|
||||||
|
|
||||||
## Release evidence
|
|
||||||
|
|
||||||
EDU PR #4 merged after its review and CI checks. Main-push CI run 1652 passed
|
|
||||||
all validation and both image builds. Deploy run 1653 passed for the exact main
|
|
||||||
SHA above.
|
|
||||||
|
|
||||||
The workstation rollout completed for both Deployments. The deployment
|
|
||||||
verified `/health` and `/live` with HTTP 200, Redis AUTH, session TTL of 1058
|
|
||||||
seconds, a delivery backlog of zero, and all nine EDU vmalert rules with
|
|
||||||
matching expressions and healthy evaluation.
|
|
||||||
|
|
||||||
The images now run by digest:
|
|
||||||
|
|
||||||
- Session keeper: `sha256:998dea51aa3015fd9cabefb0f53b030157a650c3bef72e02fe84f17d5762613d`
|
|
||||||
- Webinar checker: `sha256:92f3c1fa2bb7f9b4680a9fc76a5b33dfbea8ef3dd9c6490ebc45876fd4c54461`
|
|
||||||
|
|
||||||
Redis StatefulSet was unchanged. PVC `redis-data-pvc` remains bound to PV
|
|
||||||
`pvc-a4f2a79a-363a-4c12-ae91-92cdfc2a0d2e` with capacity 1 GiB. The existing
|
|
||||||
runtime Secret and Fernet key were preserved during the handoff. Notification
|
|
||||||
delivery was verified before closeout, as confirmed by the operator. The
|
|
||||||
deployment did not record downtime.
|
|
||||||
|
|
||||||
The release rollback snapshot is
|
|
||||||
`/home/forust/.local/state/edu-master-deploy/20261007T180541Z-4f2b2a0e37dc11ac2c75441a15076c178e219d37`.
|
|
||||||
The handoff data snapshot remains at
|
|
||||||
`/home/forust/.local/state/edu-master-deploy/handoff-20261007T080838Z`.
|
|
||||||
Both snapshots are outside Git. Do not restore old Redis data unless recovery
|
|
||||||
requires it. Never delete or recreate the Redis PVC.
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
# actionlint configuration. Passed explicitly from the ci workflow:
|
|
||||||
# actionlint -config-file .gitea/actionlint.yaml .gitea/workflows/*.yaml
|
|
||||||
#
|
|
||||||
# The self-hosted act_runner registers custom labels that actionlint cannot know
|
|
||||||
# about, so declare them here instead of silencing the whole runner-label check.
|
|
||||||
self-hosted-runner:
|
|
||||||
labels:
|
|
||||||
- arch
|
|
||||||
- homelab
|
|
||||||
- homelab-pr
|
|
||||||
- prod
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
{
|
|
||||||
"postgres": ["authentik", "gitea", "immich", "n8n", "netbox", "netronome"]
|
|
||||||
}
|
|
||||||
@@ -1,181 +0,0 @@
|
|||||||
# Homelab CI/CD
|
|
||||||
|
|
||||||
The native Gitea runners run on **vps**; production runs on **workstation**.
|
|
||||||
Main-branch checks and image builds use `homelab:host`. Pull request and
|
|
||||||
non-main checks use `homelab-pr:host` under a separate account without Docker
|
|
||||||
access. The `homelab-pr` runner is registered at User scope for `forust`, so
|
|
||||||
any repository under that account can schedule jobs that request this label.
|
|
||||||
Each runner accepts one job at a time; the build waits for every check to pass.
|
|
||||||
CI and deploy runs also show a summary with
|
|
||||||
the release SHA, image build or reuse results, deploy mode, selected services,
|
|
||||||
and image digests. Failed runs keep a summary of completed image builds, stage
|
|
||||||
results, apply results, and recorded Kubernetes recovery. The final deploy
|
|
||||||
summary is in the smoke job; earlier jobs show the state observed at that time.
|
|
||||||
Apply success is separate from health and recovery. Update the installed
|
|
||||||
workstation controller with `setup-workstation.sh` when no deploy is running.
|
|
||||||
No job images or Kubernetes credentials are needed on the VPS. Builds use one
|
|
||||||
pinned BuildKit helper container. CI and deploy are separate workflows.
|
|
||||||
|
|
||||||
## Runner installation
|
|
||||||
|
|
||||||
Install Docker Engine with Compose and Buildx, Git, Python 3.11+, Bash, curl,
|
|
||||||
GNU tar/xz, flock and systemd using the host's package manager. Keep the existing
|
|
||||||
Gitea runner 3.0.2 binary at `/usr/local/bin/gitea-runner`.
|
|
||||||
|
|
||||||
From this checkout on the VPS:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
sudo bash .gitea/runner/setup-runner.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
The installer reuses `/var/lib/gitea-runner/.runner` and the existing service.
|
|
||||||
For a new host, install the same runner binary and register as `gitea-runner`
|
|
||||||
using the registration token interactively, label `homelab:host`, and working
|
|
||||||
directory `/var/lib/gitea-runner`; then rerun the installer. Tokens never belong
|
|
||||||
in this repository or command-line examples.
|
|
||||||
|
|
||||||
Pinned tools live in the runner user's `~/.cache/homelab-ci`; CI repairs version
|
|
||||||
drift there. Installations are locked. Buildx uses only the `homelab-ci` builder,
|
|
||||||
pushes directly to the registry, and caps retained local cache at 1 GiB with a
|
|
||||||
2 GiB free-space target. This is not a hard limit on peak build disk usage.
|
|
||||||
Nothing runs `docker system prune`, removes unrelated images, or deletes volumes.
|
|
||||||
|
|
||||||
### Pull request runner
|
|
||||||
|
|
||||||
Install the unprivileged host runner on the VPS:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
sudo bash .gitea/runner/setup-pr-runner.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
Get a registration token from the user Actions runner settings. Run the
|
|
||||||
installer in a terminal. It asks for the token without echoing it, registers the
|
|
||||||
runner as `homelab-pr` with label `homelab-pr:host`, then enables the service.
|
|
||||||
The work directory is `/var/lib/gitea-pr-runner`. Confirm that Gitea lists the
|
|
||||||
runner as User scope before merging the workflow change. An unmatched label can
|
|
||||||
fall back to the default job image.
|
|
||||||
|
|
||||||
Renovate PR validation uses `pull_request_target`, which reads the workflow from
|
|
||||||
the base branch. It checks out the PR head only after runner selection and runs
|
|
||||||
that code on `homelab-pr`. Keep this workflow read-only and do not add secrets.
|
|
||||||
|
|
||||||
The PR runner has a separate home and tool cache. Do not add it to the `docker`
|
|
||||||
group or give it access to `/var/run/docker.sock`. It runs repository code from
|
|
||||||
pull requests, so keep its registration and permissions separate from the
|
|
||||||
trusted `homelab` runner. This separates users and host permissions, but both
|
|
||||||
runners still share the VPS kernel and network. Use a disposable VM if PRs from
|
|
||||||
untrusted external authors must be fully isolated.
|
|
||||||
|
|
||||||
## Workstation setup
|
|
||||||
|
|
||||||
As the existing SSH deploy user on workstation:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
sudo loginctl enable-linger forust
|
|
||||||
bash .gitea/runner/setup-workstation.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
The controller uses `/srv/homelab` as the persistent configuration tree and makes
|
|
||||||
a detached source worktree for each SHA. It never resets `/srv/homelab`, moves
|
|
||||||
local configuration, renames Compose projects, or changes volume names.
|
|
||||||
The installer records the current Kubernetes context and cluster UID in
|
|
||||||
`~/.config/homelab-deploy/environment`. Check these before installing.
|
|
||||||
|
|
||||||
Configure Gitea Actions Variables:
|
|
||||||
|
|
||||||
- `DEPLOY_HOST`, `DEPLOY_USER`, `DEPLOY_PORT`: the existing VPS-to-workstation SSH endpoint.
|
|
||||||
- `DEPLOY_KNOWN_HOSTS`: workstation's verified SSH host key entry for that endpoint.
|
|
||||||
- `AUTODEPLOY`: `false` initially; `true` enables deployment after successful main CI.
|
|
||||||
|
|
||||||
Keep `DEPLOY_SSH_KEY`, `REGISTRY_USERNAME` and `REGISTRY_PASSWORD` in Actions
|
|
||||||
Secrets. Legacy endpoint secrets remain accepted during migration. The Actions
|
|
||||||
token must have repository read and Actions read access for release downloads.
|
|
||||||
The deploy user's existing Docker registry authentication remains necessary.
|
|
||||||
|
|
||||||
## Releases and deployment
|
|
||||||
|
|
||||||
CI publishes `release-<full SHA>` as a Gitea artifact with all three owned image
|
|
||||||
digests and build input fingerprints. Unchanged images are reused only from a
|
|
||||||
successful main CI artifact, never from `:prod`. Expired artifacts cause CI to
|
|
||||||
rebuild images; they block deployment until CI is rerun.
|
|
||||||
|
|
||||||
Run deploy from main with `deploy_ref=main` or a checked SHA:
|
|
||||||
|
|
||||||
- `full`: required for the first baseline; reconcile all active components.
|
|
||||||
- `changed`: compare with the last fully successful production deploy.
|
|
||||||
- `plan`: validate configuration and show selection without changing production resources.
|
|
||||||
- `refresh_images=true`: explicitly refresh mutable third-party Compose tags.
|
|
||||||
|
|
||||||
The manual and automatic paths both require successful CI, a successful build
|
|
||||||
job and the exact SHA's release artifact. PRs cannot publish images or deploy.
|
|
||||||
Removed resources are reported and require explicit removal; no automatic prune.
|
|
||||||
Service dependencies are listed in `.gitea/deploy-dependencies.json`.
|
|
||||||
|
|
||||||
A workstation user systemd service holds the deploy lock across validation,
|
|
||||||
sequential apply, verification and smoke checks. SSH clients only submit/follow:
|
|
||||||
disconnecting or cancelling the Actions client does not kill production apply.
|
|
||||||
Retrying the same run ID does not start another apply. `ExecStopPost` recovers
|
|
||||||
interrupted runs before the unit finishes. Kubernetes rolls back to captured
|
|
||||||
revisions; configuration and persistent data are not reverted.
|
|
||||||
|
|
||||||
## Status and recovery
|
|
||||||
|
|
||||||
`--retry` repeats failed verification and smoke checks, never apply. Recovery
|
|
||||||
keeps a failed deploy out of the successful baseline, even after rollback.
|
|
||||||
|
|
||||||
On workstation (replace the numeric ID with Actions run ID and attempt):
|
|
||||||
|
|
||||||
```sh
|
|
||||||
python3 ~/.local/lib/homelab-deploy/controller.py status 123-1
|
|
||||||
python3 ~/.local/lib/homelab-deploy/controller.py recover 123-1 --retry
|
|
||||||
journalctl --user -u homelab-deploy@123-1
|
|
||||||
```
|
|
||||||
|
|
||||||
Runs live in `~/.local/state/homelab-deploy/runs`. Compose stores resolved configs
|
|
||||||
with restricted permissions; these may contain credentials and must never be
|
|
||||||
uploaded as CI artifacts. Stage logs print the exact manual recovery command
|
|
||||||
using `compose-before/<stack>.json`, the original project directory and project
|
|
||||||
name. Compose does not automatically roll back, and Nextcloud AIO's child
|
|
||||||
containers remain managed by AIO. Preserve its own backups for data recovery.
|
|
||||||
|
|
||||||
The controller retains twenty successful/planned runs and preserves failures.
|
|
||||||
Update the workstation dispatcher only when no deploy is running.
|
|
||||||
|
|
||||||
## Validation and migration rollback
|
|
||||||
|
|
||||||
```sh
|
|
||||||
python3 -m unittest discover -s tests -v
|
|
||||||
bash .gitea/tests/deploy-validation.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
Test on a separate namespace before the initial production `full` run. Check a
|
|
||||||
failed rollout, interrupted SSH and repeated run ID, and verify that an isolated
|
|
||||||
service change does not upgrade unrelated Helm releases or Compose stacks.
|
|
||||||
|
|
||||||
To roll back the migration, disable autodeploy and finish or recover the remote
|
|
||||||
run first. Restore the runner config/unit from `.before-<timestamp>` backups,
|
|
||||||
reload systemd and restart the runner. Restore the prior workflows from Git.
|
|
||||||
Production data and persistent volumes stay where they were. Do not remove run
|
|
||||||
state or Compose recovery files until recovery is confirmed.
|
|
||||||
|
|
||||||
### Compose configuration recovery
|
|
||||||
|
|
||||||
Successful deploys save the complete resolved Compose configuration in
|
|
||||||
`~/.local/state/homelab-deploy/compose-configs/`. These files can contain secrets.
|
|
||||||
Keep them private and do not commit or upload them.
|
|
||||||
The next deploy uses this configuration for its recovery file, including old
|
|
||||||
commands, environment, mounts, ports, and removed services. The recovery command
|
|
||||||
uses `--remove-orphans` to remove services added by the failed deploy. It does
|
|
||||||
not restore volume data or reverse database migrations.
|
|
||||||
|
|
||||||
On the first run after this update, the controller can use the Compose file
|
|
||||||
from the previous successful run. If that file is absent, it reads the persistent
|
|
||||||
checkout and checks its service configuration hashes against existing containers.
|
|
||||||
A mismatch stops preflight. Restore the previous configuration before retrying.
|
|
||||||
Update the installed controller with `bash .gitea/runner/setup-workstation.sh`
|
|
||||||
from the reviewed checkout before using this change.
|
|
||||||
|
|
||||||
New namespaces are checked during preflight. Server validation of their resources
|
|
||||||
runs after namespace creation and before application resources are applied.
|
|
||||||
Plan mode does not create namespaces. A failed deferred check can leave an empty
|
|
||||||
namespace; inspect it before removing it.
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
[worker.oci]
|
|
||||||
gc = true
|
|
||||||
reservedSpace = "256MB"
|
|
||||||
maxUsedSpace = "1GB"
|
|
||||||
minFreeSpace = "2GB"
|
|
||||||
|
|
||||||
[[worker.oci.gcpolicy]]
|
|
||||||
reservedSpace = "256MB"
|
|
||||||
maxUsedSpace = "1GB"
|
|
||||||
minFreeSpace = "2GB"
|
|
||||||
all = true
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
runner:
|
|
||||||
file: /var/lib/gitea-runner/.runner
|
|
||||||
capacity: 1
|
|
||||||
timeout: 5h
|
|
||||||
labels:
|
|
||||||
- homelab:host
|
|
||||||
cache:
|
|
||||||
enabled: false
|
|
||||||
container:
|
|
||||||
docker_host: unix:///var/run/docker.sock
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=Gitea Actions runner
|
|
||||||
After=network-online.target docker.service
|
|
||||||
Wants=network-online.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
User=gitea-runner
|
|
||||||
Group=gitea-runner
|
|
||||||
SupplementaryGroups=docker
|
|
||||||
WorkingDirectory=/var/lib/gitea-runner
|
|
||||||
Environment=PATH=/var/lib/gitea-runner/.cache/homelab-ci/bin:/usr/local/bin:/usr/bin:/bin
|
|
||||||
ExecStart=/usr/local/bin/gitea-runner daemon --config /etc/gitea-runner/config.yaml
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=5
|
|
||||||
UMask=0077
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=Homelab deploy %i
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=exec
|
|
||||||
EnvironmentFile=%h/.config/homelab-deploy/environment
|
|
||||||
ExecStart=/usr/bin/python3 %h/.local/lib/homelab-deploy/controller.py execute %i
|
|
||||||
ExecStopPost=/usr/bin/python3 %h/.local/lib/homelab-deploy/controller.py recover %i
|
|
||||||
RuntimeMaxSec=5h
|
|
||||||
TimeoutStopSec=135min
|
|
||||||
KillMode=control-group
|
|
||||||
UMask=0077
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
runner:
|
|
||||||
file: /var/lib/gitea-pr-runner/.runner
|
|
||||||
capacity: 1
|
|
||||||
timeout: 5h
|
|
||||||
labels:
|
|
||||||
- homelab-pr:host
|
|
||||||
cache:
|
|
||||||
enabled: false
|
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=Gitea Actions untrusted pull request runner
|
|
||||||
After=network-online.target
|
|
||||||
Wants=network-online.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
User=gitea-pr-runner
|
|
||||||
Group=gitea-pr-runner
|
|
||||||
WorkingDirectory=/var/lib/gitea-pr-runner
|
|
||||||
Environment=HOME=/var/lib/gitea-pr-runner
|
|
||||||
Environment=PATH=/var/lib/gitea-pr-runner/.cache/homelab-ci/bin:/usr/local/bin:/usr/bin:/bin
|
|
||||||
ExecStart=/usr/local/bin/gitea-runner daemon --config /etc/gitea-pr-runner/config.yaml
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=5
|
|
||||||
NoNewPrivileges=yes
|
|
||||||
PrivateTmp=yes
|
|
||||||
ProtectSystem=full
|
|
||||||
ProtectHome=yes
|
|
||||||
ProtectKernelTunables=yes
|
|
||||||
ProtectKernelModules=yes
|
|
||||||
ProtectControlGroups=yes
|
|
||||||
RestrictSUIDSGID=yes
|
|
||||||
LockPersonality=yes
|
|
||||||
UMask=0077
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Install a native runner for untrusted PR jobs without Docker access.
|
|
||||||
set -euo pipefail
|
|
||||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
[ "$(id -u)" -eq 0 ] || { echo 'Run with sudo on the runner host' >&2; exit 1; }
|
|
||||||
for tool in cp cut date getent id install runuser systemctl useradd; do
|
|
||||||
command -v "$tool" >/dev/null || { echo "Install missing prerequisite: $tool" >&2; exit 1; }
|
|
||||||
done
|
|
||||||
command -v /usr/local/bin/gitea-runner >/dev/null || {
|
|
||||||
echo 'Install gitea-runner 3.0.2 at /usr/local/bin/gitea-runner first' >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
id gitea-pr-runner >/dev/null 2>&1 || \
|
|
||||||
useradd --system --create-home --home-dir /var/lib/gitea-pr-runner --shell /usr/bin/bash gitea-pr-runner
|
|
||||||
runner_home="$(getent passwd gitea-pr-runner | cut -d: -f6)"
|
|
||||||
[ "$runner_home" = /var/lib/gitea-pr-runner ] || {
|
|
||||||
echo 'Unexpected PR runner home; inspect the existing service first' >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
case " $(id -nG gitea-pr-runner) " in
|
|
||||||
*' docker '*)
|
|
||||||
echo 'The PR runner account must not belong to the docker group' >&2
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
install -d -m 0755 /etc/gitea-pr-runner
|
|
||||||
stamp="$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
for existing in /etc/gitea-pr-runner/config.yaml /etc/systemd/system/gitea-pr-runner.service; do
|
|
||||||
[ ! -f "$existing" ] || cp -p "$existing" "$existing.before-$stamp"
|
|
||||||
done
|
|
||||||
install -m 0644 "$here/pr-config.yaml" /etc/gitea-pr-runner/config.yaml
|
|
||||||
install -m 0644 "$here/pr-runner.service" /etc/systemd/system/gitea-pr-runner.service
|
|
||||||
|
|
||||||
if [ ! -f /var/lib/gitea-pr-runner/.runner ]; then
|
|
||||||
read -r -s -p 'Enter the Gitea repository runner registration token: ' runner_token
|
|
||||||
printf '\n'
|
|
||||||
[ -n "$runner_token" ] || { echo 'Runner token is required' >&2; exit 1; }
|
|
||||||
export GITEA_RUNNER_REGISTRATION_TOKEN="$runner_token"
|
|
||||||
unset runner_token
|
|
||||||
runuser --preserve-environment -u gitea-pr-runner -- \
|
|
||||||
/usr/local/bin/gitea-runner register \
|
|
||||||
--config /etc/gitea-pr-runner/config.yaml \
|
|
||||||
--instance https://gitea.forust.xyz \
|
|
||||||
--name homelab-pr \
|
|
||||||
--labels homelab-pr:host \
|
|
||||||
--no-interactive
|
|
||||||
unset GITEA_RUNNER_REGISTRATION_TOKEN
|
|
||||||
fi
|
|
||||||
chmod 0600 /var/lib/gitea-pr-runner/.runner
|
|
||||||
|
|
||||||
systemctl daemon-reload
|
|
||||||
systemctl enable --now gitea-pr-runner.service
|
|
||||||
systemctl restart gitea-pr-runner.service
|
|
||||||
echo "PR runner ready. Configuration backups: *.before-$stamp"
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Native host runner, with pinned user-space tools and no extra CI images.
|
|
||||||
set -euo pipefail
|
|
||||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
[ "$(id -u)" -eq 0 ] || { echo 'Run with sudo on the runner host' >&2; exit 1; }
|
|
||||||
for tool in docker curl python3 git tar xz flock runuser systemctl; do
|
|
||||||
command -v "$tool" >/dev/null || { echo "Install missing prerequisite: $tool" >&2; exit 1; }
|
|
||||||
done
|
|
||||||
docker info >/dev/null
|
|
||||||
docker compose version >/dev/null
|
|
||||||
docker buildx version >/dev/null
|
|
||||||
id gitea-runner >/dev/null 2>&1 || useradd --system --create-home --home-dir /var/lib/gitea-runner --shell /usr/bin/bash gitea-runner
|
|
||||||
# Reuse the established service account and runner registration.
|
|
||||||
runner_home="$(getent passwd gitea-runner | cut -d: -f6)"
|
|
||||||
[ "$runner_home" = /var/lib/gitea-runner ] || { echo 'Unexpected runner home; inspect the existing service first' >&2; exit 1; }
|
|
||||||
runuser -u gitea-runner -- docker info >/dev/null || { echo "The runner user needs access to Docker before setup" >&2; exit 1; }
|
|
||||||
command -v gitea-runner >/dev/null || { echo 'Install gitea-runner 3.0.2 at /usr/local/bin/gitea-runner first' >&2; exit 1; }
|
|
||||||
mkdir -p /etc/gitea-runner
|
|
||||||
stamp="$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
for existing in /etc/gitea-runner/config.yaml /etc/systemd/system/gitea-runner.service; do
|
|
||||||
[ ! -f "$existing" ] || cp -p "$existing" "$existing.before-$stamp"
|
|
||||||
done
|
|
||||||
scratch="$(mktemp -d)"
|
|
||||||
trap 'rm -rf "$scratch"' EXIT
|
|
||||||
chmod 755 "$scratch"
|
|
||||||
install -m 0644 "$here/../workflows/install-ci-tools.sh" "$here/../workflows/tool-versions.env" "$scratch/"
|
|
||||||
runuser -u gitea-runner -- bash "$scratch/install-ci-tools.sh"
|
|
||||||
install -m 0644 "$here/config.yaml" /etc/gitea-runner/config.yaml
|
|
||||||
python3 - <<'PYLABELS'
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
registration = Path('/var/lib/gitea-runner/.runner')
|
|
||||||
if registration.exists():
|
|
||||||
labels = json.loads(registration.read_text()).get('labels', [])
|
|
||||||
labels = [label for label in labels if isinstance(label, str) and label.split(':')[0] != 'homelab']
|
|
||||||
labels.append('homelab:host')
|
|
||||||
config = Path('/etc/gitea-runner/config.yaml')
|
|
||||||
config.write_text(config.read_text().replace(' - homelab:host', '\n'.join(' - ' + json.dumps(label) for label in labels)))
|
|
||||||
PYLABELS
|
|
||||||
install -m 0644 "$here/gitea-runner.service" /etc/systemd/system/gitea-runner.service
|
|
||||||
if [ ! -f /var/lib/gitea-runner/.runner ]; then
|
|
||||||
echo 'Register once as gitea-runner with homelab:host before starting the service.'
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
systemctl daemon-reload
|
|
||||||
systemctl enable --now gitea-runner.service
|
|
||||||
systemctl restart gitea-runner.service
|
|
||||||
echo "Runner ready. Configuration backups: *.before-$stamp"
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Run as the existing deploy user on workstation. Never resets the working tree.
|
|
||||||
set -euo pipefail
|
|
||||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
repo="${HOMELAB_REPO:-/srv/homelab}"
|
|
||||||
for tool in python3 git kubectl helm docker flock timeout; do
|
|
||||||
command -v "$tool" >/dev/null || { echo "Install missing dependency: $tool" >&2; exit 1; }
|
|
||||||
done
|
|
||||||
[ -d "$repo/.git" ] || { echo "Missing deploy checkout: $repo" >&2; exit 1; }
|
|
||||||
[[ "$repo" =~ ^/[A-Za-z0-9_./-]+$ ]] || { echo 'Deploy path must be absolute and contain no whitespace' >&2; exit 1; }
|
|
||||||
if [ "$(loginctl show-user "$USER" -p Linger --value)" != yes ]; then
|
|
||||||
echo "Run once: sudo loginctl enable-linger $USER" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
config="${XDG_CONFIG_HOME:-$HOME/.config}/homelab-deploy"
|
|
||||||
mkdir -p "$config" "$HOME/.local/lib/homelab-deploy" "$HOME/.config/systemd/user"
|
|
||||||
chmod 700 "$config"
|
|
||||||
if [ ! -f "$config/environment" ]; then
|
|
||||||
context="$(kubectl config current-context)"
|
|
||||||
cluster_uid="$(kubectl get namespace kube-system -o jsonpath='{.metadata.uid}')"
|
|
||||||
printf 'HOMELAB_REPO=%s\nKUBE_CONTEXT=%s\nEXPECTED_CLUSTER_UID=%s\n' "$repo" "$context" "$cluster_uid" >"$config/environment"
|
|
||||||
chmod 600 "$config/environment"
|
|
||||||
fi
|
|
||||||
# Do not replace a dispatcher while an existing deploy uses it.
|
|
||||||
if systemctl --user list-units 'homelab-deploy@*' --state=running --no-legend | grep -q .; then
|
|
||||||
echo 'An existing deploy is running; wait before updating the controller' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
install -m 0755 "$here/../workflows/deploy-controller.py" "$HOME/.local/lib/homelab-deploy/controller.py"
|
|
||||||
install -m 0644 "$here/homelab-deploy@.service" "$HOME/.config/systemd/user/homelab-deploy@.service"
|
|
||||||
systemctl --user daemon-reload
|
|
||||||
echo 'Controller ready. Run a checked main SHA in full mode for the initial baseline.'
|
|
||||||
@@ -1,135 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Local regressions only: kubectl is mocked and Docker is used for config parsing.
|
|
||||||
set -euo pipefail
|
|
||||||
repo="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
scratch="$(mktemp -d)"
|
|
||||||
trap 'rm -rf "$scratch"' EXIT
|
|
||||||
|
|
||||||
mkdir -p "$scratch/repo/app" "$scratch/repo/postgres" "$scratch/repo/netbird" "$scratch/repo/renovate"
|
|
||||||
git -C "$scratch/repo" init -q
|
|
||||||
for file in app/compose.yaml postgres/shared-compose.yaml netbird/client.compose.yaml renovate/renovate-compose.yaml; do
|
|
||||||
touch "$scratch/repo/$file"
|
|
||||||
done
|
|
||||||
git -C "$scratch/repo" add .
|
|
||||||
# shellcheck source=../workflows/compose-lint.sh
|
|
||||||
source "$repo/.gitea/workflows/compose-lint.sh"
|
|
||||||
actual="$(cd "$scratch/repo" && compose_files)"
|
|
||||||
expected=$'app/compose.yaml\nnetbird/client.compose.yaml\npostgres/shared-compose.yaml\nrenovate/renovate-compose.yaml'
|
|
||||||
[ "$actual" = "$expected" ] || { echo 'Compose discovery missed a file' >&2; exit 1; }
|
|
||||||
|
|
||||||
cat >"$scratch/compose.yaml" <<'YAML'
|
|
||||||
services:
|
|
||||||
example:
|
|
||||||
image: busybox:1.37.0
|
|
||||||
environment:
|
|
||||||
REQUIRED: ${HOMELAB_TEST_REQUIRED:?required for this regression}
|
|
||||||
YAML
|
|
||||||
unset HOMELAB_TEST_REQUIRED
|
|
||||||
if validate_compose_file "$scratch/compose.yaml" >"$scratch/config.log" 2>&1; then
|
|
||||||
echo 'Full Compose validation accepted a missing variable' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
grep -q 'required for this regression' "$scratch/config.log"
|
|
||||||
HOMELAB_TEST_REQUIRED=present validate_compose_file "$scratch/compose.yaml"
|
|
||||||
|
|
||||||
cat >"$scratch/resources.json" <<'JSON'
|
|
||||||
{"kind":"List","items":[
|
|
||||||
{"kind":"Deployment","metadata":{"namespace":"app"},"spec":{"template":{"spec":{
|
|
||||||
"containers":[{"envFrom":[{"secretRef":{"name":"credentials"}},{"secretRef":{"name":"optional","optional":true}}],"env":[{"valueFrom":{"secretKeyRef":{"name":"credentials","key":"password"}}}]}],
|
|
||||||
"initContainers":[{"envFrom":[{"secretRef":{"name":"init"}}]}],
|
|
||||||
"imagePullSecrets":[{"name":"registry"}],
|
|
||||||
"volumes":[{"secret":{"secretName":"mounted"}},{"projected":{"sources":[{"secret":{"name":"projected"}},{"secret":{"name":"optional-projected","optional":true}}]}}]
|
|
||||||
}}}},
|
|
||||||
{"kind":"CronJob","metadata":{},"spec":{"jobTemplate":{"spec":{"template":{"spec":{"containers":[{"envFrom":[{"secretRef":{"name":"cron"}}]}]}}}}}},
|
|
||||||
{"kind":"IngressRoute","metadata":{"namespace":"app"},"spec":{"tls":{"secretName":"controller-issued-tls"}}}
|
|
||||||
]}
|
|
||||||
JSON
|
|
||||||
actual="$(jq -r -f "$repo/.gitea/workflows/secret-references.jq" "$scratch/resources.json" | sort)"
|
|
||||||
expected=$'app credentials\napp init\napp mounted\napp projected\napp registry\ndefault cron'
|
|
||||||
[ "$actual" = "$expected" ] || { echo "Unexpected Secret references: $actual" >&2; exit 1; }
|
|
||||||
|
|
||||||
REPO="$repo"
|
|
||||||
# shellcheck source=../workflows/deploy-lib.sh
|
|
||||||
source "$repo/.gitea/workflows/deploy-lib.sh"
|
|
||||||
K8S_MANIFESTS=("$scratch/resources.json")
|
|
||||||
KUSTOMIZE_APPS=()
|
|
||||||
# No live cluster access. Reject credentials in app even if they exist elsewhere.
|
|
||||||
kubectl() {
|
|
||||||
case "$1" in
|
|
||||||
create) cat "$scratch/resources.json" ;;
|
|
||||||
get)
|
|
||||||
if [ "$3" = credentials ] && [ "$5" = app ]; then
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
return 0
|
|
||||||
;;
|
|
||||||
*) echo "Unexpected kubectl invocation: $*" >&2; return 1 ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
if check_referenced_secrets >"$scratch/secrets.log"; then
|
|
||||||
echo 'Namespace-scoped Secret check accepted a missing Secret' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
grep -q 'MISSING OR UNREADABLE: app/credentials' "$scratch/secrets.log"
|
|
||||||
# API/rendering errors must not produce an empty reference list and pass.
|
|
||||||
kubectl() { return 1; }
|
|
||||||
if ! skip_uninstalled_vmagent_crd "$REPO/prometheus-stack/k8s/vmagent.yaml"; then
|
|
||||||
echo 'VMAgent preflight did not skip an uninstalled CRD' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
kubectl() { return 0; }
|
|
||||||
if skip_uninstalled_vmagent_crd "$REPO/prometheus-stack/k8s/vmagent.yaml"; then
|
|
||||||
echo 'VMAgent preflight skipped an installed CRD' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
if skip_uninstalled_vmagent_crd "$REPO/prometheus-stack/k8s/victoria.yaml"; then
|
|
||||||
echo 'VMAgent preflight skipped an unrelated manifest' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
kubectl() { return 1; }
|
|
||||||
if check_referenced_secrets >"$scratch/secrets.log"; then
|
|
||||||
echo 'Secret check accepted a failed manifest render' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
printf '%s\n' 'Deploy validation regressions passed.'
|
|
||||||
|
|
||||||
# New declared namespaces defer only their own resources during preflight.
|
|
||||||
render_selected_resources() {
|
|
||||||
cat <<'JSON'
|
|
||||||
{"apiVersion":"v1","kind":"List","items":[
|
|
||||||
{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"new"}},
|
|
||||||
{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"new-config","namespace":"new"}},
|
|
||||||
{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"existing-config","namespace":"default"}}
|
|
||||||
]}
|
|
||||||
JSON
|
|
||||||
}
|
|
||||||
kubectl() {
|
|
||||||
case "$1" in
|
|
||||||
get) printf '%s\n' '{"items":[{"metadata":{"name":"default"}}]}' ;;
|
|
||||||
apply) cat >"$scratch/server-input.json" ;;
|
|
||||||
*) return 1 ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
validate_server_resources true
|
|
||||||
jq -e '.items | length == 2 and all(.metadata.name != "new-config")' "$scratch/server-input.json" >/dev/null
|
|
||||||
if validate_server_resources false 2>"$scratch/deferred.log"; then
|
|
||||||
echo 'Post-namespace validation accepted a missing namespace' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
kubectl() {
|
|
||||||
case "$1" in
|
|
||||||
get) printf '%s\n' '{"items":[{"metadata":{"name":"default"}},{"metadata":{"name":"new"}}]}' ;;
|
|
||||||
apply) cat >"$scratch/server-input.json" ;;
|
|
||||||
*) return 1 ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
validate_server_resources false
|
|
||||||
jq -e '.items | length == 3' "$scratch/server-input.json" >/dev/null
|
|
||||||
render_selected_resources() {
|
|
||||||
printf '%s\n' '{"items":[{"kind":"ConfigMap","metadata":{"name":"bad","namespace":"undeclared"}}]}'
|
|
||||||
}
|
|
||||||
if validate_server_resources true 2>"$scratch/undeclared.log"; then
|
|
||||||
echo 'Preflight accepted an undeclared missing namespace' >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
printf '%s\n' 'Namespace validation regressions passed.'
|
|
||||||
@@ -1,520 +0,0 @@
|
|||||||
name: ci
|
|
||||||
"on":
|
|
||||||
push:
|
|
||||||
branches:
|
|
||||||
- main
|
|
||||||
pull_request: null
|
|
||||||
workflow_dispatch: null
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
actions: read
|
|
||||||
concurrency:
|
|
||||||
group: ci-${{ github.ref }}
|
|
||||||
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
|
|
||||||
jobs:
|
|
||||||
compose:
|
|
||||||
name: Compose
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Validate Compose files
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
source .gitea/workflows/compose-lint.sh
|
|
||||||
|
|
||||||
mapfile -t safe_flags < <(compose_safe_flags)
|
|
||||||
echo "docker compose config ${safe_flags[*]-}"
|
|
||||||
|
|
||||||
mapfile -t files < <(compose_files)
|
|
||||||
if [ "${#files[@]}" -eq 0 ]; then
|
|
||||||
echo "No Compose files found."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
failed=0
|
|
||||||
for f in "${files[@]}"; do
|
|
||||||
if ! out="$(validate_compose_file "$f" ${safe_flags[@]+"${safe_flags[@]}"} 2>&1)"; then
|
|
||||||
failed=1
|
|
||||||
echo "::error file=${f}::$(printf '%s' "$out" | head -1)"
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
if [ "$failed" -ne 0 ]; then
|
|
||||||
echo "Compose validation failed."
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
echo "checked ${#files[@]} Compose file(s)"
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Compose
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.source.conclusion == 'failure'
|
|
||||||
&& 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
workflows:
|
|
||||||
name: Workflows
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh actionlint shellcheck)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint Gitea Actions workflows with actionlint
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
actionlint -config-file .gitea/actionlint.yaml -color .gitea/workflows/*.yaml
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Workflows
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
shell:
|
|
||||||
name: Shell
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh shellcheck jq)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint shell scripts with ShellCheck
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
mapfile -t scripts < <(
|
|
||||||
git ls-files '*.sh' ':(glob)**/*.bash'
|
|
||||||
)
|
|
||||||
if [ "${#scripts[@]}" -eq 0 ]; then
|
|
||||||
echo "No shell scripts found."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
shellcheck --external-sources --source-path=SCRIPTDIR --severity=style "${scripts[@]}"
|
|
||||||
bash .gitea/tests/deploy-validation.sh
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Shell
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
formatting:
|
|
||||||
name: Formatting
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh prettier)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Check formatting with Prettier
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
mapfile -t prettier_files < <(
|
|
||||||
git ls-files \
|
|
||||||
| grep -E '\.(md|json|ya?ml|html|css)$' \
|
|
||||||
| grep -Ev '^(\.docs/|\.zed/|errorpages/html/|homepages/(forust_files|xdfnx_files)/)'
|
|
||||||
)
|
|
||||||
|
|
||||||
if [ "${#prettier_files[@]}" -eq 0 ]; then
|
|
||||||
echo "No Prettier-managed files found."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
prettier --check --ignore-unknown "${prettier_files[@]}"
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Formatting
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
python:
|
|
||||||
name: Python and tests
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh ruff jq)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint and format-check Python with Ruff
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
ruff check . .gitea/workflows
|
|
||||||
ruff format --check . .gitea/workflows
|
|
||||||
python3 -m unittest discover -s tests -v
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Python and tests
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
yaml:
|
|
||||||
name: YAML
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh yamllint)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint YAML syntax
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
mapfile -t yaml_files < <(
|
|
||||||
git ls-files '*.yaml' '*.yml' \
|
|
||||||
':!node_modules/**' \
|
|
||||||
':!**/.venv/**'
|
|
||||||
)
|
|
||||||
|
|
||||||
if [ "${#yaml_files[@]}" -eq 0 ]; then
|
|
||||||
echo "No YAML files found."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
yamllint -c .yamllint "${yaml_files[@]}"
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: YAML
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
dockerfiles:
|
|
||||||
name: Dockerfiles
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh hadolint)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint Dockerfiles
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
mapfile -t dockerfiles < <(
|
|
||||||
git ls-files ':(glob)**/Dockerfile' ':(glob)**/Dockerfile.*'
|
|
||||||
)
|
|
||||||
|
|
||||||
if [ "${#dockerfiles[@]}" -eq 0 ]; then
|
|
||||||
echo "No Dockerfiles found."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
hadolint -c .hadolint.yaml "${dockerfiles[@]}"
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Dockerfiles
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
kubernetes:
|
|
||||||
name: Kubernetes
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh kubeconform)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Validate Kubernetes manifests against JSON schemas
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
mapfile -t manifests < <(
|
|
||||||
git ls-files ':(glob)**/k8s/**/*.yaml' ':(glob)**/k8s/**/*.yml' \
|
|
||||||
| grep -Ev '(^|/)(kustomization\.ya?ml|.*\.example\.ya?ml|.*values\.ya?ml|patch-.*\.ya?ml)$'
|
|
||||||
)
|
|
||||||
|
|
||||||
if [ "${#manifests[@]}" -eq 0 ]; then
|
|
||||||
echo "No Kubernetes manifests found."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
kubeconform \
|
|
||||||
-strict \
|
|
||||||
-ignore-missing-schemas \
|
|
||||||
-summary \
|
|
||||||
"${manifests[@]}"
|
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Kubernetes
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
image-plan:
|
|
||||||
needs: [compose, workflows, shell, formatting, python, yaml, dockerfiles, kubernetes]
|
|
||||||
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 10
|
|
||||||
outputs:
|
|
||||||
matrix: ${{ steps.plan.outputs.matrix }}
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
id: source
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
with:
|
|
||||||
fetch-depth: 0
|
|
||||||
- name: Detect build inputs against successful CI
|
|
||||||
id: plan
|
|
||||||
env:
|
|
||||||
GITEA_TOKEN: ${{ github.token }}
|
|
||||||
run: python3 .gitea/workflows/release.py prepare --output build-plan.json
|
|
||||||
- name: Store the image plan
|
|
||||||
id: artifact
|
|
||||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
||||||
with:
|
|
||||||
name: build-plan
|
|
||||||
path: build-plan.json
|
|
||||||
if-no-files-found: error
|
|
||||||
retention-days: 30
|
|
||||||
|
|
||||||
- name: Write the plan result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Image plan
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP: >-
|
|
||||||
${{ steps.plan.conclusion == 'failure' && 'Build input detection' ||
|
|
||||||
steps.artifact.conclusion == 'failure' && 'Plan upload' ||
|
|
||||||
steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## Image plan\n\nResult: %s\n' "$SUMMARY_RESULT" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
images:
|
|
||||||
name: Image (${{ matrix.name }})
|
|
||||||
needs: [image-plan]
|
|
||||||
if: needs.image-plan.result == 'success'
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 60
|
|
||||||
strategy:
|
|
||||||
max-parallel: 1
|
|
||||||
fail-fast: false
|
|
||||||
matrix: ${{ fromJSON(needs.image-plan.outputs.matrix || '{"include":[{"name":"inactive"}]}') }}
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
id: source
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
- name: Download the checked image plan
|
|
||||||
id: inputs
|
|
||||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
|
||||||
with:
|
|
||||||
name: build-plan
|
|
||||||
- name: Build or reuse this image
|
|
||||||
id: check
|
|
||||||
env:
|
|
||||||
IMAGE_NAME: ${{ matrix.name }}
|
|
||||||
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
|
|
||||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
|
||||||
run: python3 .gitea/workflows/release.py image --image "$IMAGE_NAME" --output image.json
|
|
||||||
- name: Store the image result
|
|
||||||
id: artifact
|
|
||||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
||||||
with:
|
|
||||||
name: image-${{ matrix.name }}
|
|
||||||
path: image.json
|
|
||||||
if-no-files-found: error
|
|
||||||
retention-days: 30
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Image (${{ matrix.name }})
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP: >-
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Build or tag images' ||
|
|
||||||
steps.artifact.conclusion == 'failure' && 'Artifact upload' ||
|
|
||||||
steps.inputs.conclusion == 'failure' && 'Artifact download' ||
|
|
||||||
steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Retain the build job name required by the immutable release deployment gate.
|
|
||||||
build:
|
|
||||||
needs: [image-plan, images]
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
id: source
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
- name: Download all image results
|
|
||||||
id: inputs
|
|
||||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
|
||||||
with:
|
|
||||||
path: artifacts
|
|
||||||
- name: Pin SHA tags and write the complete release
|
|
||||||
id: check
|
|
||||||
env:
|
|
||||||
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
|
|
||||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
|
||||||
run: >-
|
|
||||||
python3 .gitea/workflows/release.py finalize
|
|
||||||
--plan artifacts/build-plan/build-plan.json
|
|
||||||
- name: Store commit release
|
|
||||||
id: artifact
|
|
||||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
||||||
with:
|
|
||||||
name: release-${{ github.sha }}
|
|
||||||
path: release.json
|
|
||||||
if-no-files-found: error
|
|
||||||
retention-days: 30
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Image release and SHA tags
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP: >-
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Build or tag images' ||
|
|
||||||
steps.artifact.conclusion == 'failure' && 'Artifact upload' ||
|
|
||||||
steps.inputs.conclusion == 'failure' && 'Artifact download' ||
|
|
||||||
steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Shared helpers for validating Compose files. Sourced both by steps in
|
|
||||||
# .gitea/workflows/ci.yaml and by deploy-lib.sh on the workstation.
|
|
||||||
#
|
|
||||||
# Two levels of checking, matching how the repo is structured:
|
|
||||||
#
|
|
||||||
# general every committed Compose file, active or not. Pure structure check:
|
|
||||||
# no ${VAR} interpolation, no .env lookup, no bind-mount path
|
|
||||||
# resolution. Disabled stacks deliberately have no .env in the repo
|
|
||||||
# and no values on the CI runner, so a full `config` run would fail on
|
|
||||||
# their `${VAR:?}` guards for reasons that have nothing to do with the
|
|
||||||
# change under review.
|
|
||||||
#
|
|
||||||
# full active stacks only, with interpolation and env-file resolution, so
|
|
||||||
# required variables and referenced files are actually resolved. Needs
|
|
||||||
# the gitignored .env files, so this only runs in the deploy workflow
|
|
||||||
# on the workstation.
|
|
||||||
#
|
|
||||||
# This file is meant to be sourced, not executed.
|
|
||||||
|
|
||||||
# All committed Compose files, including the ones deploy never starts.
|
|
||||||
compose_files() {
|
|
||||||
git ls-files \
|
|
||||||
'*compose.yaml' '*compose.yml'
|
|
||||||
}
|
|
||||||
|
|
||||||
# Prints the flags that turn `docker compose config` into the general check.
|
|
||||||
# Probed rather than hardcoded so an older Compose without --no-env-resolution
|
|
||||||
# still gets the flags it does support.
|
|
||||||
compose_safe_flags() {
|
|
||||||
local help flag
|
|
||||||
help="$(docker compose config --help 2>/dev/null || true)"
|
|
||||||
for flag in --no-interpolate --no-env-resolution --no-path-resolution; do
|
|
||||||
if printf '%s' "$help" | grep -q -- "$flag"; then
|
|
||||||
printf '%s\n' "$flag"
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# validate_compose_file <file> [extra docker compose config flags...]
|
|
||||||
validate_compose_file() {
|
|
||||||
local file="$1"
|
|
||||||
shift
|
|
||||||
docker compose -f "$file" config --quiet "$@"
|
|
||||||
}
|
|
||||||
@@ -1,164 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Resolve Compose images without changing project names or local bind paths."""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def output(*args, **kwargs):
|
|
||||||
return subprocess.check_output(args, text=True, **kwargs).strip() # noqa: S603
|
|
||||||
|
|
||||||
|
|
||||||
def resolve(reference):
|
|
||||||
if '@sha256:' in reference:
|
|
||||||
return reference
|
|
||||||
descriptor = json.loads(
|
|
||||||
output('docker', 'buildx', 'imagetools', 'inspect', reference, '--format', '{{json .Manifest}}')
|
|
||||||
)
|
|
||||||
digest = descriptor['digest']
|
|
||||||
if not re.fullmatch(r'sha256:[0-9a-f]{64}', digest):
|
|
||||||
raise ValueError(f'Invalid registry digest for {reference}')
|
|
||||||
# Strip tag only from the final path segment (registry ports are preserved).
|
|
||||||
repository = reference.rsplit('/', 1)
|
|
||||||
repository[-1] = repository[-1].split(':')[0]
|
|
||||||
return '/'.join(repository) + '@' + digest
|
|
||||||
|
|
||||||
|
|
||||||
def prepare(source_file):
|
|
||||||
config_repo = Path(os.environ['CONFIG_REPO'])
|
|
||||||
source_repo = Path(os.environ['REPO'])
|
|
||||||
directory = Path(os.environ['RUN_DIR'])
|
|
||||||
relative = source_file.relative_to(source_repo)
|
|
||||||
project_dir = config_repo / relative.parent
|
|
||||||
base = ['docker', 'compose', '--project-directory', str(project_dir), '-f', str(source_file)]
|
|
||||||
config = json.loads(output(*base, 'config', '--format', 'json', cwd=config_repo))
|
|
||||||
project = config['name']
|
|
||||||
previous_file = directory / 'previous.json'
|
|
||||||
previous = json.loads(previous_file.read_text()) if previous_file.exists() else {}
|
|
||||||
images_file = directory / 'compose-images.json'
|
|
||||||
locks = json.loads(images_file.read_text()) if images_file.exists() else previous.get('compose-images', {})
|
|
||||||
release = json.loads((directory / 'release.json').read_text())
|
|
||||||
state = Path(os.environ.get('HOMELAB_STATE', Path.home() / '.local/state/homelab-deploy'))
|
|
||||||
baseline = state / 'compose-configs' / f'{relative.parent.name}.json'
|
|
||||||
if not baseline.exists() and re.fullmatch(r'[0-9]+-[0-9]+', previous.get('run_id', '')):
|
|
||||||
baseline = state / 'runs' / previous['run_id'] / 'compose' / baseline.name
|
|
||||||
bootstrap = not baseline.exists()
|
|
||||||
if not bootstrap:
|
|
||||||
before = json.loads(baseline.read_text())
|
|
||||||
else:
|
|
||||||
# Bootstrap from the persistent configuration, never from the new source.
|
|
||||||
persistent_file = config_repo / relative
|
|
||||||
if persistent_file.exists():
|
|
||||||
before = json.loads(
|
|
||||||
output(
|
|
||||||
'docker',
|
|
||||||
'compose',
|
|
||||||
'--project-directory',
|
|
||||||
str(project_dir),
|
|
||||||
'-f',
|
|
||||||
str(persistent_file),
|
|
||||||
'config',
|
|
||||||
'--format',
|
|
||||||
'json',
|
|
||||||
cwd=config_repo,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
elif output('docker', 'ps', '-aq', '--filter', f'label=com.docker.compose.project={project}'):
|
|
||||||
raise ValueError(f'{project}: no previous Compose configuration; restore it before deploy')
|
|
||||||
else:
|
|
||||||
before = {'name': project, 'services': {}}
|
|
||||||
if before['name'] != project:
|
|
||||||
raise ValueError('Compose project name changed; manual migration is required')
|
|
||||||
for service, settings in config['services'].items():
|
|
||||||
reference = settings.get('image')
|
|
||||||
nextcloud_aio_master = project == 'nextcloud' and service == 'nextcloud-aio-mastercontainer'
|
|
||||||
if not reference or settings.get('build'):
|
|
||||||
raise ValueError(f'{project}/{service}: Compose deploy requires a published image')
|
|
||||||
image_repo = reference.split('@')[0].rsplit('/', 1)
|
|
||||||
image_repo[-1] = image_repo[-1].split(':')[0]
|
|
||||||
image_repo = '/'.join(image_repo)
|
|
||||||
# Nextcloud AIO validates the mastercontainer image reference and rejects
|
|
||||||
# a digest. Keep its configured tag so AIO can start and manage its stack.
|
|
||||||
if nextcloud_aio_master:
|
|
||||||
pinned = reference
|
|
||||||
elif image_repo in release['images']:
|
|
||||||
pinned = image_repo + '@' + release['images'][image_repo]
|
|
||||||
elif os.environ.get('REFRESH_IMAGES') != 'true' and reference in locks:
|
|
||||||
pinned = locks[reference]
|
|
||||||
else:
|
|
||||||
pinned = resolve(reference)
|
|
||||||
settings['image'] = pinned
|
|
||||||
locks[reference] = pinned
|
|
||||||
for service, settings in before['services'].items():
|
|
||||||
reference = settings['image']
|
|
||||||
image_repo = reference.split('@')[0].rsplit('/', 1)
|
|
||||||
image_repo[-1] = image_repo[-1].split(':')[0]
|
|
||||||
image_repo = '/'.join(image_repo)
|
|
||||||
nextcloud_aio_master = project == 'nextcloud' and service == 'nextcloud-aio-mastercontainer'
|
|
||||||
# Capture what is running, not the current value of its mutable tag.
|
|
||||||
ids = output(
|
|
||||||
'docker',
|
|
||||||
'ps',
|
|
||||||
'-aq',
|
|
||||||
'--filter',
|
|
||||||
f'label=com.docker.compose.project={project}',
|
|
||||||
'--filter',
|
|
||||||
f'label=com.docker.compose.service={service}',
|
|
||||||
).splitlines()
|
|
||||||
actual = set()
|
|
||||||
if bootstrap and ids:
|
|
||||||
expected_hash = output(
|
|
||||||
'docker',
|
|
||||||
'compose',
|
|
||||||
'--project-directory',
|
|
||||||
str(project_dir),
|
|
||||||
'-f',
|
|
||||||
str(persistent_file),
|
|
||||||
'config',
|
|
||||||
'--hash',
|
|
||||||
service,
|
|
||||||
cwd=config_repo,
|
|
||||||
).split()[-1]
|
|
||||||
for container in ids:
|
|
||||||
running_hash = output(
|
|
||||||
'docker',
|
|
||||||
'inspect',
|
|
||||||
container,
|
|
||||||
'--format',
|
|
||||||
'{{ index .Config.Labels "com.docker.compose.config-hash" }}',
|
|
||||||
)
|
|
||||||
if running_hash != expected_hash:
|
|
||||||
raise ValueError(
|
|
||||||
f'{project}/{service}: persistent config differs from running config; restore the previous config'
|
|
||||||
)
|
|
||||||
for container in ids:
|
|
||||||
image_id = output('docker', 'inspect', container, '--format', '{{.Image}}')
|
|
||||||
digests = json.loads(output('docker', 'image', 'inspect', image_id, '--format', '{{json .RepoDigests}}'))
|
|
||||||
actual.add(next((d for d in digests or [] if d.split('@')[0] == image_repo), image_id))
|
|
||||||
if len(actual) > 1:
|
|
||||||
raise ValueError(f'{project}/{service}: mixed running images, cannot capture one recovery config')
|
|
||||||
# AIO also rejects a digest in its recovery config. Preserve its tag in
|
|
||||||
# both deploy and recovery files.
|
|
||||||
if nextcloud_aio_master:
|
|
||||||
before['services'][service]['image'] = reference
|
|
||||||
else:
|
|
||||||
before['services'][service]['image'] = next(iter(actual)) if actual else reference
|
|
||||||
for name, data in (('compose', config), ('compose-before', before)):
|
|
||||||
folder = directory / name
|
|
||||||
folder.mkdir(mode=0o700, exist_ok=True)
|
|
||||||
destination = folder / f'{relative.parent.name}.json'
|
|
||||||
destination.write_text(json.dumps(data, indent=2) + '\n')
|
|
||||||
destination.chmod(0o600)
|
|
||||||
images_file.write_text(json.dumps(locks, indent=2) + '\n')
|
|
||||||
print(f'Compose {project}: images pinned; local paths preserved')
|
|
||||||
print(
|
|
||||||
f'Recovery: docker compose --project-directory {project_dir} -p {project} -f {directory}/compose-before/{relative.parent.name}.json up -d --pull never --remove-orphans'
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
prepare(Path(sys.argv[1]))
|
|
||||||
@@ -1,427 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Durable workstation deployment controller. Install with setup-workstation.sh."""
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import contextlib
|
|
||||||
import fcntl
|
|
||||||
import importlib.util
|
|
||||||
import json
|
|
||||||
import math
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
STATE = Path(os.environ.get('HOMELAB_STATE', Path.home() / '.local/state/homelab-deploy'))
|
|
||||||
CONFIG_REPO = Path(os.environ.get('HOMELAB_REPO', '/srv/homelab'))
|
|
||||||
RUN_ID = re.compile(r'[0-9]+-[0-9]+')
|
|
||||||
|
|
||||||
|
|
||||||
def command(*args, **kwargs):
|
|
||||||
return subprocess.check_output(args, text=True, **kwargs).strip() # noqa: S603, S607
|
|
||||||
|
|
||||||
|
|
||||||
def atomic_json(path, data):
|
|
||||||
temporary = path.with_suffix('.tmp')
|
|
||||||
temporary.write_text(json.dumps(data, indent=2) + '\n')
|
|
||||||
temporary.chmod(0o600)
|
|
||||||
temporary.replace(path)
|
|
||||||
|
|
||||||
|
|
||||||
@contextlib.contextmanager
|
|
||||||
def lock(name):
|
|
||||||
STATE.mkdir(mode=0o700, parents=True, exist_ok=True)
|
|
||||||
with (STATE / name).open('a') as stream:
|
|
||||||
fcntl.flock(stream, fcntl.LOCK_EX)
|
|
||||||
yield
|
|
||||||
|
|
||||||
|
|
||||||
def load_module(name, path):
|
|
||||||
spec = importlib.util.spec_from_file_location(name, path)
|
|
||||||
module = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(module)
|
|
||||||
return module
|
|
||||||
|
|
||||||
|
|
||||||
def run_directory(run_id):
|
|
||||||
if not RUN_ID.fullmatch(run_id):
|
|
||||||
raise ValueError('Run ID must be numeric workflow-id and attempt')
|
|
||||||
return STATE / 'runs' / run_id
|
|
||||||
|
|
||||||
|
|
||||||
def start(run_id):
|
|
||||||
payload = sys.stdin.buffer.read(256 * 1024 + 1)
|
|
||||||
if len(payload) > 256 * 1024:
|
|
||||||
raise ValueError('Deploy request exceeds 256 KiB')
|
|
||||||
request = json.loads(payload)
|
|
||||||
sha = request['release']['sha']
|
|
||||||
if not re.fullmatch(r'[0-9a-f]{40}', sha) or request['mode'] not in ('changed', 'full', 'plan'):
|
|
||||||
raise ValueError('Invalid deploy SHA or mode')
|
|
||||||
if not isinstance(request['refresh_images'], bool):
|
|
||||||
raise ValueError('refresh_images must be boolean')
|
|
||||||
directory = run_directory(run_id)
|
|
||||||
with lock('prepare.lock'):
|
|
||||||
if (directory / 'request.json').exists():
|
|
||||||
if json.loads((directory / 'request.json').read_text()) != request:
|
|
||||||
raise ValueError('Run ID already belongs to a different request')
|
|
||||||
else:
|
|
||||||
directory.mkdir(mode=0o700, parents=True, exist_ok=True)
|
|
||||||
command('git', '-C', str(CONFIG_REPO), 'fetch', '--quiet', 'origin', 'main')
|
|
||||||
command('git', '-C', str(CONFIG_REPO), 'merge-base', '--is-ancestor', sha, 'origin/main')
|
|
||||||
if not (directory / 'source').exists():
|
|
||||||
command('git', '-C', str(CONFIG_REPO), 'worktree', 'add', '--detach', str(directory / 'source'), sha)
|
|
||||||
if command('git', '-C', str(directory / 'source'), 'rev-parse', 'HEAD') != sha:
|
|
||||||
raise ValueError('Prepared source does not match deploy SHA')
|
|
||||||
release_module = load_module('release', directory / 'source/.gitea/workflows/release.py')
|
|
||||||
release_module.validate_release(request['release'], sha)
|
|
||||||
atomic_json(directory / 'release.json', request['release'])
|
|
||||||
atomic_json(directory / 'request.json', request)
|
|
||||||
if not (directory / 'status.json').exists():
|
|
||||||
atomic_json(directory / 'status.json', {'state': 'queued', 'stages': {}})
|
|
||||||
# Starting an existing active or finished ID is idempotent; never re-apply it.
|
|
||||||
if json.loads((directory / 'status.json').read_text())['state'] == 'queued':
|
|
||||||
command('systemctl', '--user', 'start', '--no-block', f'homelab-deploy@{run_id}.service')
|
|
||||||
print(f'Accepted deploy {run_id} ({sha})')
|
|
||||||
|
|
||||||
|
|
||||||
def environment(directory):
|
|
||||||
request = json.loads((directory / 'request.json').read_text())
|
|
||||||
return {
|
|
||||||
**os.environ,
|
|
||||||
'REPO': str(directory / 'source'),
|
|
||||||
'CONFIG_REPO': str(CONFIG_REPO),
|
|
||||||
'RUN_DIR': str(directory),
|
|
||||||
'DEPLOY_SHA': request['release']['sha'],
|
|
||||||
'RELEASE_FILE': str(directory / 'release.json'),
|
|
||||||
'DEPLOY_PLAN': str(directory / 'plan.json'),
|
|
||||||
'DEPLOY_SNAPSHOT_DIR': str(directory / 'snapshot'),
|
|
||||||
'REFRESH_IMAGES': str(request['refresh_images']).lower(),
|
|
||||||
'ROLLOUT_PARALLELISM': '4',
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def stage(directory, name, budget):
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
if name in status['stages'] and status['stages'][name].get('result') in ('success', 'failure'):
|
|
||||||
return status['stages'][name]['result'] == 'success'
|
|
||||||
started = time.time()
|
|
||||||
status['stages'][name] = {'result': 'running', 'started': started}
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
script = directory / 'source/.gitea/workflows/deploy-stage.sh'
|
|
||||||
with (directory / f'{name}.log').open('a') as log:
|
|
||||||
# timeout kills the whole stage process group, including children, before recovery.
|
|
||||||
result = subprocess.run( # noqa: S603, S607
|
|
||||||
[
|
|
||||||
shutil.which('timeout') or '/usr/bin/timeout',
|
|
||||||
'--signal=TERM',
|
|
||||||
'--kill-after=30s',
|
|
||||||
str(budget),
|
|
||||||
'bash',
|
|
||||||
str(script),
|
|
||||||
name,
|
|
||||||
],
|
|
||||||
env=environment(directory),
|
|
||||||
stdout=log,
|
|
||||||
stderr=subprocess.STDOUT,
|
|
||||||
check=False,
|
|
||||||
).returncode
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
status['stages'][name].update(
|
|
||||||
result='success' if result == 0 else 'failure', exit_code=result, seconds=round(time.time() - started)
|
|
||||||
)
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
return result == 0
|
|
||||||
|
|
||||||
|
|
||||||
def make_plan(directory):
|
|
||||||
source = directory / 'source'
|
|
||||||
planner = load_module('deploy_plan', source / '.gitea/workflows/deploy-plan.py')
|
|
||||||
request = json.loads((directory / 'request.json').read_text())
|
|
||||||
previous = json.loads((STATE / 'last-success.json').read_text()) if (STATE / 'last-success.json').exists() else None
|
|
||||||
# Helm 4 lists every release status by default and removed the --all flag.
|
|
||||||
helm = json.loads(command('helm', 'list', '-A', '-o', 'json'))
|
|
||||||
plan = planner.make_plan(source, CONFIG_REPO, request['release'], previous, request['mode'], helm)
|
|
||||||
if request['refresh_images']:
|
|
||||||
plan['selected']['compose'] = plan['active']['compose']
|
|
||||||
atomic_json(directory / 'plan.json', plan)
|
|
||||||
if previous:
|
|
||||||
atomic_json(directory / 'previous.json', previous)
|
|
||||||
# Local config is deliberately separate from the immutable Git source.
|
|
||||||
return plan
|
|
||||||
|
|
||||||
|
|
||||||
def finish_success(directory, plan):
|
|
||||||
# Repeating finalization after a crash is safe while holding deploy.lock.
|
|
||||||
plan['run_id'] = directory.name
|
|
||||||
path = directory / 'compose-images.json'
|
|
||||||
previous = directory / 'previous.json'
|
|
||||||
plan['compose-images'] = (
|
|
||||||
json.loads(path.read_text())
|
|
||||||
if path.exists()
|
|
||||||
else json.loads(previous.read_text()).get('compose-images', {})
|
|
||||||
if previous.exists()
|
|
||||||
else {}
|
|
||||||
)
|
|
||||||
configs = STATE / 'compose-configs'
|
|
||||||
configs.mkdir(mode=0o700, exist_ok=True)
|
|
||||||
for config in (directory / 'compose').glob('*.json'):
|
|
||||||
atomic_json(configs / config.name, json.loads(config.read_text()))
|
|
||||||
atomic_json(STATE / 'last-success.json', plan)
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
status['state'] = 'success'
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
try:
|
|
||||||
retain_completed(directory)
|
|
||||||
except (OSError, subprocess.CalledProcessError) as error:
|
|
||||||
print(f'Retention deferred: {error}', flush=True)
|
|
||||||
|
|
||||||
|
|
||||||
def recover(directory, retry=False):
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
if status['state'] in ('success', 'planned'):
|
|
||||||
return
|
|
||||||
completed = ('doctor', 'validate', 'apply-k8s', 'apply-compose', 'verify-k8s', 'smoke')
|
|
||||||
if all(status['stages'].get(name, {}).get('result') == 'success' for name in completed):
|
|
||||||
finish_success(directory, json.loads((directory / 'plan.json').read_text()))
|
|
||||||
return
|
|
||||||
if retry:
|
|
||||||
for name in ('verify-k8s', 'smoke'):
|
|
||||||
if status['stages'].get(name, {}).get('result') == 'failure':
|
|
||||||
del status['stages'][name]
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
snapshot = directory / 'snapshot/current'
|
|
||||||
if snapshot.exists():
|
|
||||||
stage(directory, 'verify-k8s', 7200)
|
|
||||||
stage(directory, 'smoke', 600)
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
status['state'] = 'failure'
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
|
|
||||||
|
|
||||||
def execute(run_id):
|
|
||||||
directory = run_directory(run_id)
|
|
||||||
with lock('deploy.lock'):
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
if status['state'] != 'queued':
|
|
||||||
return
|
|
||||||
# A crashed predecessor must be recovered before another apply begins.
|
|
||||||
for other in (STATE / 'runs').iterdir():
|
|
||||||
if (
|
|
||||||
other != directory
|
|
||||||
and (other / 'status.json').exists()
|
|
||||||
and json.loads((other / 'status.json').read_text())['state'] == 'running'
|
|
||||||
):
|
|
||||||
raise ValueError(f'Interrupted deploy {other.name}; run recover first')
|
|
||||||
status['state'] = 'running'
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
phase = 'plan'
|
|
||||||
try:
|
|
||||||
plan = make_plan(directory)
|
|
||||||
print(
|
|
||||||
json.dumps({'selected': plan['selected'], 'helm': plan['helm'], 'manual_removals': plan['removed']}),
|
|
||||||
flush=True,
|
|
||||||
)
|
|
||||||
phase = 'doctor'
|
|
||||||
if not stage(directory, 'doctor', 600):
|
|
||||||
raise RuntimeError('Preflight failed')
|
|
||||||
phase = 'validate'
|
|
||||||
if not stage(directory, 'validate', 1200):
|
|
||||||
raise RuntimeError('Validation failed')
|
|
||||||
if json.loads((directory / 'request.json').read_text())['mode'] == 'plan':
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
status['state'] = 'planned'
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
return
|
|
||||||
# Budget includes both rollout checks and rollback waves, plus API overhead.
|
|
||||||
phase = 'Recovery budget'
|
|
||||||
count = int(
|
|
||||||
command(
|
|
||||||
'bash',
|
|
||||||
str(directory / 'source/.gitea/workflows/deploy-stage.sh'),
|
|
||||||
'workload-count',
|
|
||||||
env=environment(directory),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
verify_budget = max(600, 2 * math.ceil(count / 4) * 300 + 120)
|
|
||||||
if verify_budget > 7200:
|
|
||||||
raise ValueError('More than two hours of recovery required; split this deploy')
|
|
||||||
phase = 'apply-k8s'
|
|
||||||
k8s_ok = stage(directory, 'apply-k8s', 2700)
|
|
||||||
phase = 'apply-compose'
|
|
||||||
compose_ok = stage(directory, 'apply-compose', 1800) if k8s_ok else False
|
|
||||||
phase = 'verify-k8s'
|
|
||||||
verify_ok = stage(directory, 'verify-k8s', verify_budget)
|
|
||||||
phase = 'smoke'
|
|
||||||
smoke_ok = stage(directory, 'smoke', 600)
|
|
||||||
if not all((k8s_ok, compose_ok, verify_ok, smoke_ok)):
|
|
||||||
raise RuntimeError('Deploy failed; inspect stage logs and recovery report')
|
|
||||||
phase = 'Save the successful baseline'
|
|
||||||
finish_success(directory, plan)
|
|
||||||
except Exception as error:
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
status['failure_stage'] = next(
|
|
||||||
(name for name, result in status['stages'].items() if result.get('result') == 'failure'), phase
|
|
||||||
)
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
with (directory / 'controller.log').open('a') as stream:
|
|
||||||
stream.write(f'{error}\n')
|
|
||||||
recover(directory)
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def retain_completed(current):
|
|
||||||
finished = []
|
|
||||||
for directory in (STATE / 'runs').iterdir():
|
|
||||||
status_file = directory / 'status.json'
|
|
||||||
if status_file.exists() and json.loads(status_file.read_text())['state'] in ('success', 'planned'):
|
|
||||||
finished.append(directory)
|
|
||||||
for directory in sorted(finished, key=lambda p: p.stat().st_mtime, reverse=True)[20:]:
|
|
||||||
if directory == current:
|
|
||||||
continue
|
|
||||||
command('git', '-C', str(CONFIG_REPO), 'worktree', 'remove', '--force', str(directory / 'source'))
|
|
||||||
shutil.rmtree(directory)
|
|
||||||
|
|
||||||
|
|
||||||
def follow(run_id, phase):
|
|
||||||
directory = run_directory(run_id)
|
|
||||||
groups = {
|
|
||||||
'apply': ('doctor', 'validate', 'apply-k8s', 'apply-compose'),
|
|
||||||
'verify': ('verify-k8s',),
|
|
||||||
'smoke': ('smoke',),
|
|
||||||
}
|
|
||||||
names = groups[phase]
|
|
||||||
offsets = {}
|
|
||||||
while True:
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
for name in (*names, 'controller'):
|
|
||||||
path = directory / f'{name}.log'
|
|
||||||
if path.exists():
|
|
||||||
with path.open() as stream:
|
|
||||||
stream.seek(offsets.get(name, 0))
|
|
||||||
content = stream.read()
|
|
||||||
if content:
|
|
||||||
print(content, end='', flush=True)
|
|
||||||
offsets[name] = stream.tell()
|
|
||||||
stages = status['stages']
|
|
||||||
if all(stages.get(name, {}).get('result') in ('success', 'failure') for name in names):
|
|
||||||
return all(stages[name]['result'] == 'success' for name in names)
|
|
||||||
if status['state'] in ('success', 'failure', 'planned'):
|
|
||||||
return status['state'] in ('success', 'planned')
|
|
||||||
time.sleep(3)
|
|
||||||
|
|
||||||
|
|
||||||
def summary(run_id):
|
|
||||||
directory = run_directory(run_id)
|
|
||||||
request = json.loads((directory / 'request.json').read_text())
|
|
||||||
release = request['release']
|
|
||||||
plan_file = directory / 'plan.json'
|
|
||||||
lines = [
|
|
||||||
f'## Deploy `{release["sha"]}`',
|
|
||||||
'',
|
|
||||||
f'- Mode: `{request["mode"]}`',
|
|
||||||
f'- Refresh third-party images: `{request["refresh_images"]}`',
|
|
||||||
]
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
if status.get('failure_stage'):
|
|
||||||
lines.append(f'- Failed stage: **{status["failure_stage"]}**')
|
|
||||||
lines.extend(
|
|
||||||
[
|
|
||||||
'',
|
|
||||||
f'- Observed run state: **{status["state"]}**',
|
|
||||||
'',
|
|
||||||
'### Stage results',
|
|
||||||
'| Stage | Result | Exit code |',
|
|
||||||
'| --- | --- | --- |',
|
|
||||||
]
|
|
||||||
)
|
|
||||||
for name in ('doctor', 'validate', 'apply-k8s', 'apply-compose', 'verify-k8s', 'smoke'):
|
|
||||||
stage_result = status['stages'].get(name, {})
|
|
||||||
lines.append(f'| {name} | {stage_result.get("result", "not started")} | {stage_result.get("exit_code", "—")} |')
|
|
||||||
lines.extend(['', '### Apply and Helm recovery results'])
|
|
||||||
events_file = directory / 'apply-events.jsonl'
|
|
||||||
events = []
|
|
||||||
if events_file.exists():
|
|
||||||
for line in events_file.read_text().splitlines():
|
|
||||||
try:
|
|
||||||
events.append(json.loads(line))
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
lines.append('- An operation record is incomplete. Check the stage log.')
|
|
||||||
latest = {(event['action'], event['target']): event['result'] for event in events}
|
|
||||||
lines.extend(f'- `{action}` `{target}`: **{result}**' for (action, target), result in latest.items())
|
|
||||||
if not latest:
|
|
||||||
lines.append('- No apply results were recorded.')
|
|
||||||
lines.append('- A completed apply does not confirm health. See verification and smoke results.')
|
|
||||||
lines.extend(['', '### Kubernetes recovery'])
|
|
||||||
pointer = directory / 'snapshot/current'
|
|
||||||
failed = Path(pointer.read_text().strip()) / 'failed-workloads' if pointer.exists() else None
|
|
||||||
if failed and failed.exists():
|
|
||||||
contents = failed.read_text()
|
|
||||||
counts = dict(re.findall(r'^(ROLLED_BACK|UNRECOVERED)=([0-9]+)$', contents, re.MULTILINE))
|
|
||||||
if not contents.strip():
|
|
||||||
lines.append('- No failed workloads were recorded. See the verification result above.')
|
|
||||||
elif counts:
|
|
||||||
lines.append(f'- Workloads restored: **{counts.get("ROLLED_BACK", "unknown")}**')
|
|
||||||
lines.append(f'- Workloads that need manual recovery: **{counts.get("UNRECOVERED", "unknown")}**')
|
|
||||||
else:
|
|
||||||
lines.append('- Rollback has no recorded result yet. Check the verification log.')
|
|
||||||
else:
|
|
||||||
lines.append('- No workload rollback was recorded. This does not confirm health.')
|
|
||||||
lines.append('- Compose requires manual recovery. Use the saved command in the apply log.')
|
|
||||||
if not plan_file.exists():
|
|
||||||
lines.extend(['', 'Plan was not created. Check the controller log.'])
|
|
||||||
print('\n'.join(lines))
|
|
||||||
return
|
|
||||||
plan = json.loads(plan_file.read_text())
|
|
||||||
lines.extend(['', '### Selected services'])
|
|
||||||
count = 0
|
|
||||||
for kind, services in plan['selected'].items():
|
|
||||||
for service in services:
|
|
||||||
lines.append(f'- `{kind}`: `{service}`')
|
|
||||||
count += 1
|
|
||||||
if not count:
|
|
||||||
lines.append('- None')
|
|
||||||
lines.extend(['', '### Selected Helm releases'])
|
|
||||||
lines.extend(f'- `{release}`' for release in plan.get('helm', []))
|
|
||||||
if not plan.get('helm'):
|
|
||||||
lines.append('- None')
|
|
||||||
lines.extend(['', '### Images pinned in the checked release'])
|
|
||||||
lines.extend(f'- `{image}@{digest}`' for image, digest in sorted(release['images'].items()))
|
|
||||||
lines.extend(['', '### Removed resources requiring manual review'])
|
|
||||||
lines.extend(f'- `{item}`' for item in plan.get('removed', []))
|
|
||||||
if not plan.get('removed'):
|
|
||||||
lines.append('- None')
|
|
||||||
print('\n'.join(lines))
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
os.umask(0o077)
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument('action', choices=('start', 'execute', 'recover', 'status', 'follow', 'summary'))
|
|
||||||
parser.add_argument('run_id')
|
|
||||||
parser.add_argument('phase', nargs='?', choices=('apply', 'verify', 'smoke'))
|
|
||||||
parser.add_argument('--retry', action='store_true', help='Retry failed recovery checks; never repeat apply')
|
|
||||||
args = parser.parse_args()
|
|
||||||
directory = run_directory(args.run_id)
|
|
||||||
if args.action == 'start':
|
|
||||||
start(args.run_id)
|
|
||||||
elif args.action == 'execute':
|
|
||||||
execute(args.run_id)
|
|
||||||
elif args.action == 'recover':
|
|
||||||
with lock('deploy.lock'):
|
|
||||||
recover(directory, retry=args.retry)
|
|
||||||
elif args.action == 'status':
|
|
||||||
print((directory / 'status.json').read_text())
|
|
||||||
if (directory / 'plan.json').exists():
|
|
||||||
plan = json.loads((directory / 'plan.json').read_text())
|
|
||||||
print(json.dumps({k: plan[k] for k in ('sha', 'selected', 'helm', 'removed')}, indent=2))
|
|
||||||
elif args.action == 'summary':
|
|
||||||
summary(args.run_id)
|
|
||||||
elif not follow(args.run_id, args.phase):
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -1,124 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Calculate selected components against the last fully successful deploy."""
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def output(*args, **kwargs):
|
|
||||||
return subprocess.check_output(args, text=True, **kwargs).strip() # noqa: S603
|
|
||||||
|
|
||||||
|
|
||||||
def tracked(repo):
|
|
||||||
return output('git', '-C', str(repo), 'ls-files').splitlines()
|
|
||||||
|
|
||||||
|
|
||||||
def helm_releases(repo):
|
|
||||||
text = (repo / '.gitea/workflows/deploy-lib.sh').read_text()
|
|
||||||
return [line.split('|') for line in re.findall(r'^ "([^"\n]+\|[^"\n]+)"$', text, re.MULTILINE)]
|
|
||||||
|
|
||||||
|
|
||||||
def inventory(repo):
|
|
||||||
files = tracked(repo)
|
|
||||||
k8s = sorted(
|
|
||||||
{f.split('/k8s/')[0] for f in files if '/k8s/' in f and (repo / f.split('/k8s/')[0] / 'k8s/active').is_file()}
|
|
||||||
)
|
|
||||||
compose = sorted(
|
|
||||||
{
|
|
||||||
str(Path(f).parent)
|
|
||||||
for f in files
|
|
||||||
if Path(f).name in ('compose.yaml', 'compose.yml') and (repo / Path(f).parent / 'active').is_file()
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return {'k8s': k8s, 'compose': compose}
|
|
||||||
|
|
||||||
|
|
||||||
def file_hash(path):
|
|
||||||
return hashlib.sha256(path.read_bytes()).hexdigest() if path.is_file() else 'missing'
|
|
||||||
|
|
||||||
|
|
||||||
def make_plan(repo, config_repo, release, previous, mode, live_helm):
|
|
||||||
active = inventory(repo)
|
|
||||||
all_services = set(active['k8s'] + active['compose'])
|
|
||||||
helm_inputs = {}
|
|
||||||
helm_selected = []
|
|
||||||
for name, chart, namespace, version, values, marker in helm_releases(repo):
|
|
||||||
if not (repo / marker).is_file():
|
|
||||||
continue
|
|
||||||
value_path = repo / values if (repo / values).is_file() else config_repo / values
|
|
||||||
if not value_path.is_file():
|
|
||||||
raise ValueError(f'Missing Helm values: {values}')
|
|
||||||
stamp = hashlib.sha256(f'{chart}|{version}|{file_hash(value_path)}'.encode()).hexdigest()
|
|
||||||
helm_inputs[name] = stamp
|
|
||||||
live = next((h for h in live_helm if h['name'] == name and h['namespace'] == namespace), None)
|
|
||||||
if (
|
|
||||||
mode == 'full'
|
|
||||||
or previous is None
|
|
||||||
or previous.get('helm_inputs', {}).get(name) != stamp
|
|
||||||
or live is None
|
|
||||||
or live.get('status') != 'deployed'
|
|
||||||
or live.get('chart') != f'{chart.split("/")[-1]}-{version}'
|
|
||||||
):
|
|
||||||
helm_selected.append(name)
|
|
||||||
local_inputs = {}
|
|
||||||
for service in all_services:
|
|
||||||
candidates = [config_repo / service / '.env']
|
|
||||||
if service in active['compose']:
|
|
||||||
candidates.append(config_repo / '.env')
|
|
||||||
cfg = config_repo / service / 'config'
|
|
||||||
if cfg.is_dir():
|
|
||||||
candidates.extend(
|
|
||||||
p for p in cfg.rglob('*') if p.is_file() and p.suffix in ('.yaml', '.yml', '.json', '.conf')
|
|
||||||
)
|
|
||||||
local_inputs[service] = hashlib.sha256(
|
|
||||||
'\n'.join(f'{p.relative_to(config_repo)}:{file_hash(p)}' for p in sorted(candidates)).encode()
|
|
||||||
).hexdigest()
|
|
||||||
if previous is None:
|
|
||||||
if mode == 'changed':
|
|
||||||
raise ValueError('No successful baseline; run deploy in full mode first')
|
|
||||||
changed = set(all_services)
|
|
||||||
removed = []
|
|
||||||
else:
|
|
||||||
paths = output('git', '-C', str(repo), 'diff', '--name-only', previous['sha'], release['sha']).splitlines()
|
|
||||||
changed = {service for service in all_services for path in paths if path.startswith(service + '/')}
|
|
||||||
if any(path.startswith('.gitea/') for path in paths):
|
|
||||||
changed |= all_services
|
|
||||||
changed |= {s for s in all_services if previous.get('local_inputs', {}).get(s) != local_inputs[s]}
|
|
||||||
for file in tracked(repo):
|
|
||||||
owners = {service for service in all_services if file.startswith(service + '/')}
|
|
||||||
if not owners or not file.endswith(('.yaml', '.yml')):
|
|
||||||
continue
|
|
||||||
text = (repo / file).read_text()
|
|
||||||
if any(
|
|
||||||
image in text and previous.get('images', {}).get(image) != digest
|
|
||||||
for image, digest in release['images'].items()
|
|
||||||
):
|
|
||||||
changed |= owners
|
|
||||||
removed = sorted(
|
|
||||||
set(previous.get('active', {}).get('k8s', []) + previous.get('active', {}).get('compose', []))
|
|
||||||
- all_services
|
|
||||||
)
|
|
||||||
removed += [path for path in paths if '/k8s/' in path and not (repo / path).exists()]
|
|
||||||
if mode == 'full':
|
|
||||||
changed = set(all_services)
|
|
||||||
dependencies = json.loads((repo / '.gitea/deploy-dependencies.json').read_text())
|
|
||||||
while True:
|
|
||||||
expanded = changed | {dependent for service in changed for dependent in dependencies.get(service, [])}
|
|
||||||
if expanded == changed:
|
|
||||||
break
|
|
||||||
changed = expanded
|
|
||||||
return {
|
|
||||||
'version': 1,
|
|
||||||
'sha': release['sha'],
|
|
||||||
'images': release['images'],
|
|
||||||
'active': active,
|
|
||||||
'selected': {kind: sorted(set(services) & changed) for kind, services in active.items()},
|
|
||||||
'helm': helm_selected,
|
|
||||||
'helm_inputs': helm_inputs,
|
|
||||||
'local_inputs': local_inputs,
|
|
||||||
'removed': sorted(set(removed)),
|
|
||||||
'full_smoke': mode == 'full' or 'traefik' in changed,
|
|
||||||
}
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
source "${REPO:?}/.gitea/workflows/deploy-lib.sh"
|
|
||||||
case "${1:?stage required}" in
|
|
||||||
workload-count)
|
|
||||||
select_manifests >/dev/null
|
|
||||||
selected_workload_refs | sort -u | wc -l
|
|
||||||
;;
|
|
||||||
*) run_stage "$1" ;;
|
|
||||||
esac
|
|
||||||
@@ -1,140 +0,0 @@
|
|||||||
name: deploy
|
|
||||||
|
|
||||||
on:
|
|
||||||
workflow_run:
|
|
||||||
workflows: [ci]
|
|
||||||
branches: [main]
|
|
||||||
types: [completed]
|
|
||||||
workflow_dispatch:
|
|
||||||
inputs:
|
|
||||||
deploy_ref:
|
|
||||||
description: "Commit already checked by successful main CI (main or SHA)"
|
|
||||||
default: main
|
|
||||||
required: true
|
|
||||||
deploy_mode:
|
|
||||||
description: "First deploy requires full; plan changes no production resources"
|
|
||||||
type: choice
|
|
||||||
options: [changed, full, plan]
|
|
||||||
default: changed
|
|
||||||
refresh_images:
|
|
||||||
description: "Explicitly refresh mutable third-party Compose tags"
|
|
||||||
type: boolean
|
|
||||||
default: false
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
actions: read
|
|
||||||
|
|
||||||
concurrency:
|
|
||||||
group: deploy-main
|
|
||||||
cancel-in-progress: false
|
|
||||||
|
|
||||||
env:
|
|
||||||
DEPLOY_HOST: ${{ vars.DEPLOY_HOST || secrets.DEPLOY_HOST }}
|
|
||||||
DEPLOY_PORT: ${{ vars.DEPLOY_PORT || secrets.DEPLOY_PORT }}
|
|
||||||
DEPLOY_USER: ${{ vars.DEPLOY_USER || secrets.DEPLOY_USER }}
|
|
||||||
DEPLOY_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
|
|
||||||
DEPLOY_KNOWN_HOSTS: ${{ vars.DEPLOY_KNOWN_HOSTS }}
|
|
||||||
DEPLOY_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt || 1 }}
|
|
||||||
DEPLOY_MODE: ${{ inputs.deploy_mode || 'changed' }}
|
|
||||||
REFRESH_IMAGES: ${{ inputs.refresh_images && 'true' || 'false' }}
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
gate:
|
|
||||||
if: >-
|
|
||||||
github.ref == 'refs/heads/main' &&
|
|
||||||
(vars.AUTODEPLOY == 'true' || github.event_name == 'workflow_dispatch') &&
|
|
||||||
(github.event_name != 'workflow_run' ||
|
|
||||||
(github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'main'))
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 10
|
|
||||||
outputs:
|
|
||||||
sha: ${{ steps.release.outputs.sha }}
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
||||||
with:
|
|
||||||
fetch-depth: 0
|
|
||||||
- name: Check successful CI and download the exact commit release
|
|
||||||
id: release
|
|
||||||
env:
|
|
||||||
GITEA_TOKEN: ${{ github.token }}
|
|
||||||
DEPLOY_REF: ${{ inputs.deploy_ref || 'main' }}
|
|
||||||
EVENT_SHA: ${{ github.event.workflow_run.head_sha }}
|
|
||||||
run: python3 .gitea/workflows/release.py gate --ref "$DEPLOY_REF" --event-sha "$EVENT_SHA"
|
|
||||||
- name: Submit durable deploy to workstation
|
|
||||||
run: bash .gitea/workflows/ssh-run.sh start
|
|
||||||
- name: Write the request result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
REQUEST_RESULT: ${{ job.status }}
|
|
||||||
CHECKED_SHA: ${{ steps.release.outputs.sha }}
|
|
||||||
run: |
|
|
||||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## Deploy request\n\n- Result: **%s**\n- Checked commit: %s\n- Mode: %s\n' "$REQUEST_RESULT" "${CHECKED_SHA:-not checked}" "$DEPLOY_MODE" >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
if [ "$REQUEST_RESULT" != success ]; then
|
|
||||||
echo 'Open the failed step log. If SSH submission failed, check the remote controller state.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
apply:
|
|
||||||
needs: [gate]
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 120
|
|
||||||
steps:
|
|
||||||
- name: Checkout checked commit
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
||||||
with:
|
|
||||||
ref: ${{ needs.gate.outputs.sha }}
|
|
||||||
- name: Follow validation and sequential Kubernetes / Compose apply
|
|
||||||
run: bash .gitea/workflows/ssh-run.sh apply
|
|
||||||
- name: Write the deploy result
|
|
||||||
if: always()
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/ssh-run.sh ]; then
|
|
||||||
bash .gitea/workflows/ssh-run.sh summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
echo 'Source checkout failed. The remote deploy state is unknown. Check the job log.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
|
|
||||||
verify:
|
|
||||||
needs: [gate, apply]
|
|
||||||
if: always() && needs.gate.result == 'success'
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 130
|
|
||||||
steps:
|
|
||||||
- name: Checkout checked commit
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
||||||
with:
|
|
||||||
ref: ${{ needs.gate.outputs.sha }}
|
|
||||||
- name: Follow workload verification and recovery
|
|
||||||
run: bash .gitea/workflows/ssh-run.sh verify
|
|
||||||
- name: Write the deploy result
|
|
||||||
if: always()
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/ssh-run.sh ]; then
|
|
||||||
bash .gitea/workflows/ssh-run.sh summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
echo 'Source checkout failed. The remote deploy state is unknown. Check the job log.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
|
|
||||||
smoke:
|
|
||||||
needs: [gate, verify]
|
|
||||||
if: always() && needs.gate.result == 'success'
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout checked commit
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
||||||
with:
|
|
||||||
ref: ${{ needs.gate.outputs.sha }}
|
|
||||||
- name: Follow public route checks
|
|
||||||
run: bash .gitea/workflows/ssh-run.sh smoke
|
|
||||||
- name: Write the deploy result
|
|
||||||
if: always()
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/ssh-run.sh ]; then
|
|
||||||
bash .gitea/workflows/ssh-run.sh summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
echo 'Source checkout failed. The remote deploy state is unknown. Check the job log.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
@@ -1,285 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Installs the pinned CI tools into "$TOOLS_DIR/bin" and echoes that directory
|
|
||||||
# on stdout, so callers can do:
|
|
||||||
#
|
|
||||||
# export PATH="$(bash .gitea/workflows/install-ci-tools.sh kubeconform shellcheck):$PATH"
|
|
||||||
#
|
|
||||||
# Versions come from tool-versions.env next to this script and are kept fresh by
|
|
||||||
# Renovate. Re-running is cheap: an already-installed tool at the pinned version
|
|
||||||
# is left alone.
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
# shellcheck source=tool-versions.env
|
|
||||||
. "$here/tool-versions.env"
|
|
||||||
|
|
||||||
TOOLS_DIR="${TOOLS_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/homelab-ci}"
|
|
||||||
BIN_DIR="$TOOLS_DIR/bin"
|
|
||||||
mkdir -p "$BIN_DIR"
|
|
||||||
# A runner may accept overlapping workflows even though each workflow is sequential.
|
|
||||||
exec 9>"$TOOLS_DIR/install.lock"
|
|
||||||
flock -w 300 9
|
|
||||||
export UV_TOOL_DIR="$TOOLS_DIR/uv-tools"
|
|
||||||
export UV_CACHE_DIR="$TOOLS_DIR/uv-cache"
|
|
||||||
# The just-installed tools must resolve inside this script too: callers only
|
|
||||||
# prepend BIN_DIR to PATH after the script exits, so a bare `uv` below would
|
|
||||||
# miss the binary install_uv just placed (exit 127 on a clean runner).
|
|
||||||
export PATH="$BIN_DIR:$PATH"
|
|
||||||
|
|
||||||
arch="$(uname -m)"
|
|
||||||
# Upstream projects disagree on arch spelling: kubeconform and actionlint use
|
|
||||||
# Go names (amd64/arm64), shellcheck uses uname names (x86_64/aarch64), node
|
|
||||||
# uses neither (x64/arm64), and hadolint mixes the two in a single release
|
|
||||||
# (x86_64 but arm64).
|
|
||||||
case "$arch" in
|
|
||||||
x86_64 | amd64)
|
|
||||||
goarch=amd64
|
|
||||||
sharch=x86_64
|
|
||||||
nodearch=x64
|
|
||||||
hadolintarch=x86_64
|
|
||||||
;;
|
|
||||||
aarch64 | arm64)
|
|
||||||
goarch=arm64
|
|
||||||
sharch=aarch64
|
|
||||||
nodearch=arm64
|
|
||||||
hadolintarch=arm64
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo "install-ci-tools: unsupported architecture: $arch" >&2
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
fetch() {
|
|
||||||
# fetch <url> <dest>
|
|
||||||
if command -v curl >/dev/null 2>&1; then
|
|
||||||
curl -sSLf --connect-timeout 15 --max-time 120 --retry 3 -o "$2" "$1"
|
|
||||||
elif command -v wget >/dev/null 2>&1; then
|
|
||||||
wget -q -O "$2" "$1"
|
|
||||||
else
|
|
||||||
echo "install-ci-tools: neither curl nor wget is available" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# resolve <command>
|
|
||||||
# Absolute path to use for invoking a tool: the copy in BIN_DIR when present,
|
|
||||||
# otherwise the name for PATH lookup. Every version check and every in-script
|
|
||||||
# invocation goes through this, so a tool missing from both places reads as
|
|
||||||
# "not installed" instead of dying with 127 under `set -e`.
|
|
||||||
resolve() {
|
|
||||||
if [ -x "$BIN_DIR/$1" ]; then
|
|
||||||
printf '%s' "$BIN_DIR/$1"
|
|
||||||
else
|
|
||||||
printf '%s' "$1"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# installed_version <command>
|
|
||||||
# Prints the version of an already-installed tool, or nothing. Each tool spells
|
|
||||||
# its version flag differently, hence the case.
|
|
||||||
installed_version() {
|
|
||||||
local bin out
|
|
||||||
bin="$(resolve "$1")"
|
|
||||||
if ! command -v "$bin" >/dev/null 2>&1; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
case "$1" in
|
|
||||||
kubeconform) out="$("$bin" -v 2>/dev/null | head -1 || true)" ;;
|
|
||||||
*) out="$("$bin" --version 2>/dev/null | head -1 || true)" ;;
|
|
||||||
esac
|
|
||||||
printf '%s' "$out"
|
|
||||||
}
|
|
||||||
|
|
||||||
# at_version <command> <expected>
|
|
||||||
at_version() {
|
|
||||||
local version expected="${2#v}"
|
|
||||||
version="$(installed_version "$1")"
|
|
||||||
if [[ "$version" =~ (^|[^0-9.])v?([0-9]+(\.[0-9]+)+) ]]; then
|
|
||||||
[ "${BASH_REMATCH[2]}" = "$expected" ]
|
|
||||||
else
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
install_kubeconform() {
|
|
||||||
if at_version kubeconform "v${KUBECONFORM_VERSION}"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
local tmp
|
|
||||||
tmp="$(mktemp -d)"
|
|
||||||
fetch "https://github.com/yannh/kubeconform/releases/download/v${KUBECONFORM_VERSION}/kubeconform-linux-${goarch}.tar.gz" \
|
|
||||||
"$tmp/kubeconform.tar.gz"
|
|
||||||
tar -xzf "$tmp/kubeconform.tar.gz" -C "$tmp" kubeconform
|
|
||||||
install -m 0755 "$tmp/kubeconform" "$BIN_DIR/kubeconform"
|
|
||||||
rm -rf "$tmp"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_shellcheck() {
|
|
||||||
if at_version shellcheck "${SHELLCHECK_VERSION}"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
local tmp
|
|
||||||
tmp="$(mktemp -d)"
|
|
||||||
fetch "https://github.com/koalaman/shellcheck/releases/download/v${SHELLCHECK_VERSION}/shellcheck-v${SHELLCHECK_VERSION}.linux.${sharch}.tar.xz" \
|
|
||||||
"$tmp/shellcheck.tar.xz"
|
|
||||||
tar -xJf "$tmp/shellcheck.tar.xz" -C "$tmp" --strip-components=1 "shellcheck-v${SHELLCHECK_VERSION}/shellcheck"
|
|
||||||
install -m 0755 "$tmp/shellcheck" "$BIN_DIR/shellcheck"
|
|
||||||
rm -rf "$tmp"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_jq() {
|
|
||||||
if at_version jq "${JQ_VERSION}"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
fetch "https://github.com/jqlang/jq/releases/download/jq-${JQ_VERSION}/jq-linux-${goarch}" \
|
|
||||||
"$BIN_DIR/jq"
|
|
||||||
chmod 0755 "$BIN_DIR/jq"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_uv() {
|
|
||||||
if at_version uv "${UV_VERSION}"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
local tmp
|
|
||||||
tmp="$(mktemp -d)"
|
|
||||||
# uv release tags carry no leading v, unlike every other tool installed here.
|
|
||||||
fetch "https://github.com/astral-sh/uv/releases/download/${UV_VERSION}/uv-${sharch}-unknown-linux-gnu.tar.gz" \
|
|
||||||
"$tmp/uv.tar.gz"
|
|
||||||
tar -xzf "$tmp/uv.tar.gz" -C "$tmp" --strip-components=1 "uv-${sharch}-unknown-linux-gnu/uv"
|
|
||||||
install -m 0755 "$tmp/uv" "$BIN_DIR/uv"
|
|
||||||
rm -rf "$tmp"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_hadolint() {
|
|
||||||
if at_version hadolint "${HADOLINT_VERSION}"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
# A bare binary, no archive: hadolint ships one file per platform.
|
|
||||||
fetch "https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-linux-${hadolintarch}" \
|
|
||||||
"$BIN_DIR/hadolint"
|
|
||||||
chmod 0755 "$BIN_DIR/hadolint"
|
|
||||||
}
|
|
||||||
|
|
||||||
# ruff and yamllint both come from PyPI as wheels, which uv unpacks for us.
|
|
||||||
install_uv_tool() {
|
|
||||||
# <package> <pinned version>
|
|
||||||
if at_version "$1" "$2"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
install_uv
|
|
||||||
UV_TOOL_BIN_DIR="$BIN_DIR" "$BIN_DIR/uv" tool install --force "$1==$2" >/dev/null
|
|
||||||
}
|
|
||||||
|
|
||||||
install_ruff() {
|
|
||||||
install_uv_tool ruff "${RUFF_VERSION}"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_yamllint() {
|
|
||||||
install_uv_tool yamllint "${YAMLLINT_VERSION}"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_pip_audit() {
|
|
||||||
install_uv_tool pip-audit "${PIP_AUDIT_VERSION}"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_prettier() {
|
|
||||||
install_node
|
|
||||||
if at_version prettier "${PRETTIER_VERSION}"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
# Not a standalone binary: prettier's entry point requires ../package.json
|
|
||||||
# relative to its own real path, so the package directory has to survive
|
|
||||||
# next to it. Hence a versioned directory plus a relative symlink, rather
|
|
||||||
# than copying the one file out as the other installers do.
|
|
||||||
local dir="$BIN_DIR/prettier-${PRETTIER_VERSION}"
|
|
||||||
if [ ! -f "$dir/package/package.json" ]; then
|
|
||||||
rm -rf "$dir"
|
|
||||||
mkdir -p "$dir"
|
|
||||||
fetch "https://registry.npmjs.org/prettier/-/prettier-${PRETTIER_VERSION}.tgz" "$dir/prettier.tgz"
|
|
||||||
tar -xzf "$dir/prettier.tgz" -C "$dir"
|
|
||||||
rm -f "$dir/prettier.tgz"
|
|
||||||
# npm strips the exec bit from bin/ on the way into the tarball.
|
|
||||||
chmod 0755 "$dir/package/bin/prettier.cjs"
|
|
||||||
fi
|
|
||||||
# Relative, so the whole tree stays valid if TOOLS_DIR is relocated.
|
|
||||||
ln -sfn "prettier-${PRETTIER_VERSION}/package/bin/prettier.cjs" "$BIN_DIR/prettier"
|
|
||||||
}
|
|
||||||
|
|
||||||
install_node() {
|
|
||||||
# npm gets checked by running it, not by looking it up: what matters is that
|
|
||||||
# it answers, so a stub, a half-removed Arch package or a name that resolves
|
|
||||||
# to something broken all have to read as "not installed". The runner's npm
|
|
||||||
# is a symlink into /usr/lib/node_modules/npm, which is exactly the kind of
|
|
||||||
# thing that disappears between runs.
|
|
||||||
if at_version node "v${NODE_VERSION}" && [ -n "$(installed_version npm)" ]; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
# Same shape as prettier above: the tarball's bin/npm and bin/npx are links
|
|
||||||
# into lib/node_modules, so the whole tree has to survive next to them.
|
|
||||||
local dir="$BIN_DIR/node-${NODE_VERSION}"
|
|
||||||
if [ ! -x "$dir/bin/node" ]; then
|
|
||||||
rm -rf "$dir"
|
|
||||||
mkdir -p "$dir"
|
|
||||||
fetch "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-${nodearch}.tar.xz" \
|
|
||||||
"$dir/node.tar.xz"
|
|
||||||
tar -xJf "$dir/node.tar.xz" -C "$dir" --strip-components=1 "node-v${NODE_VERSION}-linux-${nodearch}"
|
|
||||||
rm -f "$dir/node.tar.xz"
|
|
||||||
fi
|
|
||||||
# Relative, so the whole tree stays valid if TOOLS_DIR is relocated.
|
|
||||||
for bin in node npm npx; do
|
|
||||||
ln -sfn "node-${NODE_VERSION}/bin/${bin}" "$BIN_DIR/${bin}"
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
install_actionlint() {
|
|
||||||
if at_version actionlint "${ACTIONLINT_VERSION}"; then
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
local tmp
|
|
||||||
tmp="$(mktemp -d)"
|
|
||||||
fetch "https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_${goarch}.tar.gz" \
|
|
||||||
"$tmp/actionlint.tar.gz"
|
|
||||||
tar -xzf "$tmp/actionlint.tar.gz" -C "$tmp" actionlint
|
|
||||||
install -m 0755 "$tmp/actionlint" "$BIN_DIR/actionlint"
|
|
||||||
rm -rf "$tmp"
|
|
||||||
}
|
|
||||||
|
|
||||||
main() {
|
|
||||||
wanted=("$@")
|
|
||||||
if [ "${#wanted[@]}" -eq 0 ]; then
|
|
||||||
wanted=(node jq kubeconform shellcheck actionlint prettier ruff yamllint hadolint)
|
|
||||||
fi
|
|
||||||
|
|
||||||
for tool in "${wanted[@]}"; do
|
|
||||||
case "$tool" in
|
|
||||||
kubeconform) install_kubeconform ;;
|
|
||||||
shellcheck) install_shellcheck ;;
|
|
||||||
jq) install_jq ;;
|
|
||||||
actionlint) install_actionlint ;;
|
|
||||||
prettier) install_prettier ;;
|
|
||||||
ruff) install_ruff ;;
|
|
||||||
yamllint) install_yamllint ;;
|
|
||||||
pip-audit) install_pip_audit ;;
|
|
||||||
hadolint) install_hadolint ;;
|
|
||||||
node) install_node ;;
|
|
||||||
uv) install_uv ;;
|
|
||||||
*)
|
|
||||||
echo "install-ci-tools: unknown tool: $tool" >&2
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
for old in "$BIN_DIR"/node-* "$BIN_DIR"/prettier-*; do
|
|
||||||
[ -d "$old" ] || continue
|
|
||||||
case "$(basename "$old")" in
|
|
||||||
"node-$NODE_VERSION"|"prettier-$PRETTIER_VERSION") ;;
|
|
||||||
*) rm -rf "$old" ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
if [ -x "$BIN_DIR/uv" ]; then "$BIN_DIR/uv" cache prune >/dev/null; fi
|
|
||||||
printf '%s\n' "$BIN_DIR"
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ "${BASH_SOURCE[0]}" = "$0" ]; then main "$@"; fi
|
|
||||||
@@ -1,516 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""CI release artifacts and the SHA-specific Gitea deployment gate (stdlib only)."""
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import hashlib
|
|
||||||
import io
|
|
||||||
import itertools
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
import urllib.error
|
|
||||||
import urllib.parse
|
|
||||||
import urllib.request
|
|
||||||
import zipfile
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
SHA = re.compile(r'[0-9a-f]{40}')
|
|
||||||
DIGEST = re.compile(r'sha256:[0-9a-f]{64}')
|
|
||||||
IMAGES = {
|
|
||||||
'error-pages': ('errorpages', 'errorpages/Dockerfile'),
|
|
||||||
'forust-homepage': ('homepages', 'homepages/Dockerfile.forust'),
|
|
||||||
'xdfnx-homepage': ('homepages', 'homepages/Dockerfile.xdfnx'),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def command(*args, **kwargs):
|
|
||||||
"""Arguments are passed directly to the executable, never to a shell."""
|
|
||||||
return subprocess.check_output(args, text=True, **kwargs).strip() # noqa: S603, S607
|
|
||||||
|
|
||||||
|
|
||||||
def validate_release(data, sha=None):
|
|
||||||
if data.get('version') != 1 or not SHA.fullmatch(data.get('sha', '')):
|
|
||||||
raise ValueError('Invalid release version or SHA')
|
|
||||||
if sha is not None and data['sha'] != sha:
|
|
||||||
raise ValueError('Release SHA does not match the checked CI commit')
|
|
||||||
expected = {f'gcr.forust.xyz/forust/{name}' for name in IMAGES}
|
|
||||||
if set(data.get('images', {})) != expected:
|
|
||||||
raise ValueError('Release must contain all owned images')
|
|
||||||
if not all(DIGEST.fullmatch(value) for value in data['images'].values()):
|
|
||||||
raise ValueError('Release has an invalid image digest')
|
|
||||||
if set(data.get('inputs', {})) != expected or not all(
|
|
||||||
re.fullmatch(r'[0-9a-f]{64}', value) for value in data['inputs'].values()
|
|
||||||
):
|
|
||||||
raise ValueError('Release has invalid build input fingerprints')
|
|
||||||
return data
|
|
||||||
|
|
||||||
|
|
||||||
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
|
||||||
def redirect_request(self, _req, _fp, _code, _msg, _headers, _newurl):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
class Gitea:
|
|
||||||
def __init__(self):
|
|
||||||
self.origin = os.environ['GITHUB_SERVER_URL'].rstrip('/')
|
|
||||||
if urllib.parse.urlsplit(self.origin).scheme != 'https':
|
|
||||||
raise ValueError('Gitea API must use HTTPS')
|
|
||||||
self.repository = os.environ['GITHUB_REPOSITORY']
|
|
||||||
if not re.fullmatch(r'[\w.-]+/[\w.-]+', self.repository):
|
|
||||||
raise ValueError('Invalid Gitea repository')
|
|
||||||
self.token = os.environ['GITEA_TOKEN']
|
|
||||||
self.base = f'{self.origin}/api/v1/repos/{self.repository}'
|
|
||||||
|
|
||||||
def request(self, url, *, archive=False):
|
|
||||||
if not url.startswith(self.base + '/'):
|
|
||||||
raise ValueError('Refusing to send the Actions token to another origin')
|
|
||||||
req = urllib.request.Request(url, headers={'Authorization': f'token {self.token}'}) # noqa: S310 -- HTTPS origin validated above
|
|
||||||
opener = urllib.request.build_opener(NoRedirect())
|
|
||||||
try:
|
|
||||||
response = opener.open(req, timeout=30) # noqa: S310
|
|
||||||
except urllib.error.HTTPError as error:
|
|
||||||
if not archive or error.code not in (301, 302, 303, 307, 308):
|
|
||||||
raise RuntimeError(f'Gitea API returned HTTP {error.code}') from None
|
|
||||||
target = urllib.parse.urljoin(url, error.headers['Location'])
|
|
||||||
if urllib.parse.urlsplit(target).scheme != 'https':
|
|
||||||
raise ValueError('Artifact redirect must use HTTPS') from None
|
|
||||||
# Signed storage redirects must never receive the Gitea token.
|
|
||||||
response = urllib.request.urlopen(target, timeout=30) # noqa: S310
|
|
||||||
with response:
|
|
||||||
payload = response.read(8 * 1024 * 1024 + 1)
|
|
||||||
if len(payload) > 8 * 1024 * 1024:
|
|
||||||
raise ValueError('Gitea response exceeds 8 MiB')
|
|
||||||
return payload if archive else json.loads(payload)
|
|
||||||
|
|
||||||
def pages(self, path, key, **params):
|
|
||||||
for page in range(1, 101):
|
|
||||||
query = urllib.parse.urlencode({**params, 'page': page, 'limit': 50})
|
|
||||||
data = self.request(f'{self.base}/{path}?{query}')
|
|
||||||
entries = data[key]
|
|
||||||
yield from entries
|
|
||||||
if len(entries) < 50:
|
|
||||||
return
|
|
||||||
raise RuntimeError('Gitea pagination limit exceeded')
|
|
||||||
|
|
||||||
def successful_runs(self, sha=None):
|
|
||||||
params = {'branch': 'main', 'status': 'success', 'exclude_pull_requests': 'true'}
|
|
||||||
if sha:
|
|
||||||
params['head_sha'] = sha
|
|
||||||
for run in self.pages('actions/workflows/ci.yaml/runs', 'workflow_runs', **params):
|
|
||||||
if (
|
|
||||||
run.get('status') == 'completed'
|
|
||||||
and run.get('conclusion') == 'success'
|
|
||||||
and run.get('head_branch') == 'main'
|
|
||||||
and run.get('event') in ('push', 'workflow_dispatch')
|
|
||||||
and (run.get('repository') or {}).get('full_name') == self.repository
|
|
||||||
and (run.get('head_repository') or run.get('repository') or {}).get('full_name') == self.repository
|
|
||||||
and (sha is None or run.get('head_sha') == sha)
|
|
||||||
):
|
|
||||||
yield run
|
|
||||||
|
|
||||||
def release(self, run):
|
|
||||||
sha = run['head_sha']
|
|
||||||
jobs = list(self.pages(f'actions/runs/{run["id"]}/jobs', 'jobs'))
|
|
||||||
# A green workflow with a skipped build must not authorize a deploy.
|
|
||||||
if not any(job.get('name') == 'build' and job.get('conclusion') == 'success' for job in jobs):
|
|
||||||
raise ValueError('CI build job did not succeed')
|
|
||||||
artifacts = self.request(f'{self.base}/actions/runs/{run["id"]}/artifacts')['artifacts']
|
|
||||||
matching = [a for a in artifacts if a['name'] == f'release-{sha}' and not a.get('expired')]
|
|
||||||
if len(matching) != 1:
|
|
||||||
raise ValueError('CI release artifact is missing, expired or ambiguous; rerun CI')
|
|
||||||
blob = self.request(f'{self.base}/actions/artifacts/{matching[0]["id"]}/zip', archive=True)
|
|
||||||
with zipfile.ZipFile(io.BytesIO(blob)) as archive:
|
|
||||||
files = [entry for entry in archive.infolist() if not entry.is_dir()]
|
|
||||||
if len(files) != 1 or files[0].filename != 'release.json' or files[0].file_size > 256 * 1024:
|
|
||||||
raise ValueError('Unexpected release archive contents')
|
|
||||||
return validate_release(json.loads(archive.read(files[0])), sha)
|
|
||||||
|
|
||||||
|
|
||||||
def fingerprint(context, dockerfile):
|
|
||||||
tree = command('git', 'ls-tree', '-r', 'HEAD', '--', context, dockerfile, '.gitea/workflows/release.py')
|
|
||||||
return hashlib.sha256(tree.encode()).hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def gate(output, requested_ref, event_sha):
|
|
||||||
command('git', 'fetch', '--quiet', 'origin', 'main')
|
|
||||||
if event_sha:
|
|
||||||
if not SHA.fullmatch(event_sha):
|
|
||||||
raise ValueError('Invalid workflow_run SHA')
|
|
||||||
sha = event_sha
|
|
||||||
else:
|
|
||||||
if requested_ref == 'main':
|
|
||||||
requested_ref = 'origin/main'
|
|
||||||
sha = command('git', 'rev-parse', '--verify', '--end-of-options', f'{requested_ref}^{{commit}}')
|
|
||||||
if not SHA.fullmatch(sha):
|
|
||||||
raise ValueError('Invalid deploy SHA')
|
|
||||||
command('git', 'merge-base', '--is-ancestor', sha, 'origin/main')
|
|
||||||
api = Gitea()
|
|
||||||
runs = list(api.successful_runs(sha))
|
|
||||||
if not runs:
|
|
||||||
raise ValueError(f'No successful main CI for {sha}; run CI before deploying')
|
|
||||||
release = api.release(max(runs, key=lambda run: run['id']))
|
|
||||||
output.write_text(json.dumps(release, indent=2) + '\n')
|
|
||||||
if os.environ.get('GITHUB_OUTPUT'):
|
|
||||||
with Path(os.environ['GITHUB_OUTPUT']).open('a') as stream:
|
|
||||||
stream.write(f'sha={sha}\n')
|
|
||||||
print(f'CI gate accepted {sha}')
|
|
||||||
|
|
||||||
|
|
||||||
def prepare_images(output):
|
|
||||||
sha = command('git', 'rev-parse', 'HEAD')
|
|
||||||
if sha != os.environ['GITHUB_SHA'] or not SHA.fullmatch(sha):
|
|
||||||
raise ValueError('Build checkout does not match GITHUB_SHA')
|
|
||||||
api = Gitea()
|
|
||||||
previous = None
|
|
||||||
for run in sorted(itertools.islice(api.successful_runs(), 50), key=lambda item: item['id'], reverse=True):
|
|
||||||
if str(run['id']) == os.environ.get('GITHUB_RUN_ID'):
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
previous = api.release(run)
|
|
||||||
break
|
|
||||||
except ValueError:
|
|
||||||
# Expired artifacts only cost a rebuild; mutable tags are never a fallback.
|
|
||||||
continue
|
|
||||||
targets = []
|
|
||||||
for name, (context, dockerfile) in IMAGES.items():
|
|
||||||
image = f'gcr.forust.xyz/forust/{name}'
|
|
||||||
inputs = fingerprint(context, dockerfile)
|
|
||||||
old_digest = (previous or {}).get('images', {}).get(image)
|
|
||||||
targets.append(
|
|
||||||
{
|
|
||||||
'name': name,
|
|
||||||
'image': image,
|
|
||||||
'context': context,
|
|
||||||
'dockerfile': dockerfile,
|
|
||||||
'inputs': inputs,
|
|
||||||
'reuse_digest': old_digest if (previous or {}).get('inputs', {}).get(image) == inputs else None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
output.write_text(json.dumps({'sha': sha, 'targets': targets}, indent=2) + '\n')
|
|
||||||
if os.environ.get('GITHUB_OUTPUT'):
|
|
||||||
with Path(os.environ['GITHUB_OUTPUT']).open('a') as stream:
|
|
||||||
stream.write('matrix=' + json.dumps({'include': targets}, separators=(',', ':')) + '\n')
|
|
||||||
print(f'Prepared {len(targets)} image jobs; {sum(t["reuse_digest"] is None for t in targets)} require builds')
|
|
||||||
|
|
||||||
|
|
||||||
def checked_plan(path):
|
|
||||||
data = json.loads(path.read_text())
|
|
||||||
sha = command('git', 'rev-parse', 'HEAD')
|
|
||||||
if data.get('sha') != sha or sha != os.environ['GITHUB_SHA'] or not SHA.fullmatch(sha):
|
|
||||||
raise ValueError('Image plan does not match the checked source commit')
|
|
||||||
targets = data.get('targets', [])
|
|
||||||
if sorted(t['name'] for t in targets) != sorted(IMAGES):
|
|
||||||
raise ValueError('Image plan must contain each owned image once')
|
|
||||||
for target in targets:
|
|
||||||
name = target['name']
|
|
||||||
context, dockerfile = IMAGES[name]
|
|
||||||
if (target['context'], target['dockerfile'], target['image']) != (
|
|
||||||
context,
|
|
||||||
dockerfile,
|
|
||||||
f'gcr.forust.xyz/forust/{name}',
|
|
||||||
) or target['inputs'] != fingerprint(context, dockerfile):
|
|
||||||
raise ValueError('Image plan has invalid build inputs')
|
|
||||||
if target['reuse_digest'] is not None and not DIGEST.fullmatch(target['reuse_digest']):
|
|
||||||
raise ValueError('Image plan has an invalid reuse digest')
|
|
||||||
return data
|
|
||||||
|
|
||||||
|
|
||||||
def build_images(output, report, name, plan):
|
|
||||||
data = checked_plan(plan)
|
|
||||||
sha = data['sha']
|
|
||||||
target = next(t for t in data['targets'] if t['name'] == name)
|
|
||||||
context, dockerfile = IMAGES[name]
|
|
||||||
docker_config = tempfile.mkdtemp(prefix='homelab-registry-')
|
|
||||||
builder_config = Path.home() / '.cache/homelab-ci/buildx'
|
|
||||||
builder_config.mkdir(parents=True, exist_ok=True)
|
|
||||||
env = {**os.environ, 'DOCKER_CONFIG': docker_config, 'BUILDX_CONFIG': str(builder_config)}
|
|
||||||
try:
|
|
||||||
report['phase'] = 'Registry login'
|
|
||||||
subprocess.run( # noqa: S603, S607
|
|
||||||
[
|
|
||||||
shutil.which('docker') or '/usr/bin/docker',
|
|
||||||
'login',
|
|
||||||
'gcr.forust.xyz',
|
|
||||||
'-u',
|
|
||||||
os.environ['REGISTRY_USERNAME'],
|
|
||||||
'--password-stdin',
|
|
||||||
],
|
|
||||||
input=os.environ['REGISTRY_PASSWORD'],
|
|
||||||
text=True,
|
|
||||||
check=True,
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
report['phase'] = 'Prepare the builder'
|
|
||||||
builder = 'homelab-ci'
|
|
||||||
versions = dict(
|
|
||||||
re.findall(r'^([A-Z_]+)="([^"\n]+)"$', Path('.gitea/workflows/tool-versions.env').read_text(), re.MULTILINE)
|
|
||||||
)
|
|
||||||
image = versions['BUILDKIT_IMAGE']
|
|
||||||
signature = builder_config / 'homelab-ci-image'
|
|
||||||
exists = (
|
|
||||||
subprocess.run( # noqa: S603
|
|
||||||
[shutil.which('docker') or '/usr/bin/docker', 'buildx', 'inspect', builder],
|
|
||||||
capture_output=True,
|
|
||||||
env=env,
|
|
||||||
).returncode
|
|
||||||
== 0
|
|
||||||
)
|
|
||||||
if exists and (not signature.exists() or signature.read_text().strip() != image):
|
|
||||||
command('docker', 'buildx', 'rm', '--keep-state', builder, env=env)
|
|
||||||
exists = False
|
|
||||||
if not exists:
|
|
||||||
command(
|
|
||||||
'docker',
|
|
||||||
'buildx',
|
|
||||||
'create',
|
|
||||||
'--name',
|
|
||||||
builder,
|
|
||||||
'--driver',
|
|
||||||
'docker-container',
|
|
||||||
'--driver-opt',
|
|
||||||
f'image={image}',
|
|
||||||
'--buildkitd-config',
|
|
||||||
'.gitea/runner/buildkitd.toml',
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
signature.write_text(image + '\n')
|
|
||||||
release = {'version': 1, 'sha': sha, 'images': {}, 'inputs': {}}
|
|
||||||
report['images'] = release['images']
|
|
||||||
report['phase'] = f'Build or reuse {name}'
|
|
||||||
report['current'] = name
|
|
||||||
image = f'gcr.forust.xyz/forust/{name}'
|
|
||||||
inputs = target['inputs']
|
|
||||||
old_digest = target['reuse_digest']
|
|
||||||
exists = False
|
|
||||||
if old_digest:
|
|
||||||
exists = (
|
|
||||||
subprocess.run( # noqa: S603, S607
|
|
||||||
[
|
|
||||||
shutil.which('docker') or '/usr/bin/docker',
|
|
||||||
'buildx',
|
|
||||||
'imagetools',
|
|
||||||
'inspect',
|
|
||||||
f'{image}@{old_digest}',
|
|
||||||
],
|
|
||||||
capture_output=True,
|
|
||||||
env=env,
|
|
||||||
timeout=60,
|
|
||||||
).returncode
|
|
||||||
== 0
|
|
||||||
)
|
|
||||||
if exists:
|
|
||||||
print(f'Reuse {name}: inputs unchanged')
|
|
||||||
digest = old_digest
|
|
||||||
report['reused'].append(name)
|
|
||||||
else:
|
|
||||||
print(f'Build {name}', flush=True)
|
|
||||||
metadata = Path(docker_config) / 'metadata.json'
|
|
||||||
command(
|
|
||||||
'docker',
|
|
||||||
'buildx',
|
|
||||||
'build',
|
|
||||||
'--builder',
|
|
||||||
builder,
|
|
||||||
'--platform',
|
|
||||||
'linux/amd64',
|
|
||||||
'--provenance=false',
|
|
||||||
'--cache-from',
|
|
||||||
f'type=registry,ref={image}:buildcache',
|
|
||||||
'--cache-to',
|
|
||||||
f'type=registry,ref={image}:buildcache,mode=max',
|
|
||||||
'--output',
|
|
||||||
f'type=image,name={image},push-by-digest=true,name-canonical=true,push=true',
|
|
||||||
'--metadata-file',
|
|
||||||
str(metadata),
|
|
||||||
'--file',
|
|
||||||
dockerfile,
|
|
||||||
context,
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
digest = json.loads(metadata.read_text())['containerimage.digest']
|
|
||||||
report['built'].append(name)
|
|
||||||
release['images'][image] = digest
|
|
||||||
release['inputs'][image] = inputs
|
|
||||||
if not DIGEST.fullmatch(digest):
|
|
||||||
raise ValueError('Image job returned an invalid digest')
|
|
||||||
output.write_text(json.dumps(release, indent=2) + '\n')
|
|
||||||
report['current'] = None
|
|
||||||
report['phase'] = 'Release file saved'
|
|
||||||
finally:
|
|
||||||
# Cleanup errors must neither leak credentials nor mask the original build error.
|
|
||||||
try:
|
|
||||||
subprocess.run( # noqa: S603
|
|
||||||
[
|
|
||||||
shutil.which('docker') or '/usr/bin/docker',
|
|
||||||
'buildx',
|
|
||||||
'prune',
|
|
||||||
'--builder',
|
|
||||||
'homelab-ci',
|
|
||||||
'--force',
|
|
||||||
'--max-used-space',
|
|
||||||
'1gb',
|
|
||||||
],
|
|
||||||
env=env,
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
except (OSError, subprocess.TimeoutExpired):
|
|
||||||
print('CI builder cache cleanup deferred', flush=True)
|
|
||||||
finally:
|
|
||||||
shutil.rmtree(docker_config)
|
|
||||||
|
|
||||||
|
|
||||||
def write_summary(lines):
|
|
||||||
path = os.environ.get('GITHUB_STEP_SUMMARY')
|
|
||||||
if path:
|
|
||||||
try:
|
|
||||||
with Path(path).open('a') as stream:
|
|
||||||
stream.write('\n'.join(lines) + '\n\n')
|
|
||||||
except OSError:
|
|
||||||
print('WARNING: cannot write the job summary')
|
|
||||||
|
|
||||||
|
|
||||||
def check_summary():
|
|
||||||
lines = [
|
|
||||||
f'## {os.environ["SUMMARY_CHECK"]}',
|
|
||||||
'',
|
|
||||||
f'- Commit: `{os.environ.get("GITHUB_SHA", "unknown")}`',
|
|
||||||
f'- Result: **{os.environ["SUMMARY_RESULT"]}**',
|
|
||||||
]
|
|
||||||
if os.environ.get('SUMMARY_FAILED_STEP'):
|
|
||||||
lines.append(f'- Failed step: {os.environ["SUMMARY_FAILED_STEP"]}')
|
|
||||||
if os.environ['SUMMARY_RESULT'] != 'success':
|
|
||||||
lines.append('- Open the failed step log for the error details.')
|
|
||||||
write_summary(lines)
|
|
||||||
|
|
||||||
|
|
||||||
def build(output, name, plan):
|
|
||||||
report = {'phase': 'Check the source commit', 'current': None, 'built': [], 'reused': [], 'images': {}}
|
|
||||||
result = 'failure'
|
|
||||||
try:
|
|
||||||
build_images(output, report, name, plan)
|
|
||||||
result = 'success'
|
|
||||||
finally:
|
|
||||||
lines = [
|
|
||||||
f'## Image release `{os.environ.get("GITHUB_SHA", "unknown")}`',
|
|
||||||
'',
|
|
||||||
f'- Result: **{result}**',
|
|
||||||
f'- Last stage: {report["phase"]}',
|
|
||||||
]
|
|
||||||
if result == 'failure':
|
|
||||||
lines.append('- No release from this build can be deployed. Open the failed step log.')
|
|
||||||
if report['current']:
|
|
||||||
lines.append(f'- Image at the failure: `{report["current"]}`')
|
|
||||||
for title, key in (('Built', 'built'), ('Reused from successful CI', 'reused')):
|
|
||||||
lines.extend(['', f'### {title}'])
|
|
||||||
lines.extend(f'- `{name}`' for name in report[key])
|
|
||||||
if not report[key]:
|
|
||||||
lines.append('- None')
|
|
||||||
lines.extend(['', '### Completed image digests'])
|
|
||||||
lines.extend(f'- `{image}@{digest}`' for image, digest in report['images'].items())
|
|
||||||
if not report['images']:
|
|
||||||
lines.append('- None')
|
|
||||||
write_summary(lines)
|
|
||||||
|
|
||||||
|
|
||||||
def render(stream, destination):
|
|
||||||
release = validate_release(json.loads(Path(os.environ['RELEASE_FILE']).read_text()), os.environ['DEPLOY_SHA'])
|
|
||||||
image_line = re.compile(
|
|
||||||
r"^(\s*(?:-\s*)?image:\s*)(['\"]?)(gcr\.forust\.xyz/forust/[\w.-]+)(?::[\w.-]+|@sha256:[0-9a-f]{64})\2(\s*(?:#.*)?)$"
|
|
||||||
)
|
|
||||||
rendered = []
|
|
||||||
for line in stream:
|
|
||||||
match = image_line.fullmatch(line.rstrip('\n'))
|
|
||||||
if match:
|
|
||||||
prefix, quote, image, tail = match.groups()
|
|
||||||
if image not in release['images']:
|
|
||||||
raise ValueError(f'Owned image missing from checked release: {image}')
|
|
||||||
line = f'{prefix}{quote}{image}@{release["images"][image]}{quote}{tail}\n'
|
|
||||||
elif re.match(r'\s*(?:-\s*)?image:', line) and 'gcr.forust.xyz/forust/' in line:
|
|
||||||
raise ValueError('Unsupported owned image syntax; refusing to apply a mutable tag')
|
|
||||||
rendered.append(line)
|
|
||||||
destination.writelines(rendered)
|
|
||||||
|
|
||||||
|
|
||||||
def finalize_images(output, fragments, plan):
|
|
||||||
data = checked_plan(plan)
|
|
||||||
sha = data['sha']
|
|
||||||
release = {'version': 1, 'sha': sha, 'images': {}, 'inputs': {}}
|
|
||||||
for name in IMAGES:
|
|
||||||
fragment = json.loads((fragments / f'image-{name}' / 'image.json').read_text())
|
|
||||||
image = f'gcr.forust.xyz/forust/{name}'
|
|
||||||
if fragment.get('sha') != sha or fragment.get('version') != 1 or set(fragment.get('images', {})) != {image}:
|
|
||||||
raise ValueError('Image job artifact is missing or belongs to another commit')
|
|
||||||
target = next(t for t in data['targets'] if t['name'] == name)
|
|
||||||
if fragment.get('inputs') != {image: target['inputs']}:
|
|
||||||
raise ValueError('Image artifact does not match the build plan')
|
|
||||||
release['images'].update(fragment['images'])
|
|
||||||
release['inputs'].update(fragment['inputs'])
|
|
||||||
validate_release(release, sha)
|
|
||||||
# Only a complete set of successful image jobs can publish the release tags.
|
|
||||||
docker_config = tempfile.mkdtemp(prefix='homelab-registry-')
|
|
||||||
env = {**os.environ, 'DOCKER_CONFIG': docker_config}
|
|
||||||
try:
|
|
||||||
subprocess.run( # noqa: S603, S607
|
|
||||||
[
|
|
||||||
shutil.which('docker') or '/usr/bin/docker',
|
|
||||||
'login',
|
|
||||||
'gcr.forust.xyz',
|
|
||||||
'-u',
|
|
||||||
os.environ['REGISTRY_USERNAME'],
|
|
||||||
'--password-stdin',
|
|
||||||
],
|
|
||||||
input=os.environ['REGISTRY_PASSWORD'],
|
|
||||||
text=True,
|
|
||||||
check=True,
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
for image, digest in release['images'].items():
|
|
||||||
command(
|
|
||||||
'docker',
|
|
||||||
'buildx',
|
|
||||||
'imagetools',
|
|
||||||
'create',
|
|
||||||
'--prefer-index=false',
|
|
||||||
'--tag',
|
|
||||||
f'{image}:sha-{sha}',
|
|
||||||
f'{image}@{digest}',
|
|
||||||
env=env,
|
|
||||||
timeout=90,
|
|
||||||
)
|
|
||||||
output.write_text(json.dumps(release, indent=2) + '\n')
|
|
||||||
finally:
|
|
||||||
shutil.rmtree(docker_config)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument('action', choices=('prepare', 'image', 'finalize', 'gate', 'render', 'check-summary'))
|
|
||||||
parser.add_argument('--output', type=Path, default=Path('release.json'))
|
|
||||||
parser.add_argument('--ref', default='main')
|
|
||||||
parser.add_argument('--event-sha', default='')
|
|
||||||
parser.add_argument('--image', choices=IMAGES)
|
|
||||||
parser.add_argument('--plan', type=Path, default=Path('build-plan.json'))
|
|
||||||
parser.add_argument('--fragments', type=Path, default=Path('artifacts'))
|
|
||||||
args = parser.parse_args()
|
|
||||||
if args.action == 'check-summary':
|
|
||||||
check_summary()
|
|
||||||
elif args.action == 'render':
|
|
||||||
render(sys.stdin, sys.stdout)
|
|
||||||
elif args.action == 'gate':
|
|
||||||
gate(args.output, args.ref, args.event_sha)
|
|
||||||
elif args.action == 'prepare':
|
|
||||||
prepare_images(args.output)
|
|
||||||
elif args.action == 'image':
|
|
||||||
if not args.image:
|
|
||||||
parser.error('--image is required')
|
|
||||||
build(args.output, args.image, args.plan)
|
|
||||||
else:
|
|
||||||
finalize_images(args.output, args.fragments, args.plan)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
name: renovate-ci
|
|
||||||
|
|
||||||
on:
|
|
||||||
# Read the workflow from the trusted base branch. PR code runs only on the
|
|
||||||
# unprivileged runner selected below.
|
|
||||||
pull_request_target:
|
|
||||||
paths:
|
|
||||||
- "renovate/**"
|
|
||||||
- ".gitea/workflows/renovate-ci.yaml"
|
|
||||||
- ".gitea/workflows/sync-renovate-configmap.sh"
|
|
||||||
- ".gitea/workflows/compose-lint.sh"
|
|
||||||
- ".gitea/workflows/install-ci-tools.sh"
|
|
||||||
- ".gitea/workflows/tool-versions.env"
|
|
||||||
push:
|
|
||||||
branches:
|
|
||||||
- main
|
|
||||||
paths:
|
|
||||||
- "renovate/**"
|
|
||||||
- ".gitea/workflows/renovate-ci.yaml"
|
|
||||||
- ".gitea/workflows/sync-renovate-configmap.sh"
|
|
||||||
- ".gitea/workflows/compose-lint.sh"
|
|
||||||
- ".gitea/workflows/install-ci-tools.sh"
|
|
||||||
- ".gitea/workflows/tool-versions.env"
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
validate-renovate:
|
|
||||||
runs-on: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 20
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
||||||
with:
|
|
||||||
ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || github.sha }}
|
|
||||||
|
|
||||||
# renovate/k8s/cronjob.yaml is the single source of truth for the version.
|
|
||||||
- name: Resolve the deployed Renovate version
|
|
||||||
id: image
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
image="$(sed -n 's|.*image:[[:space:]]*\(renovate/renovate:[^[:space:]]*\).*|\1|p' \
|
|
||||||
renovate/k8s/cronjob.yaml | head -1)"
|
|
||||||
if [[ ! "$image" =~ ^renovate/renovate:([0-9]+\.[0-9]+\.[0-9]+)$ ]]; then
|
|
||||||
echo "::error::expected a pinned renovate/renovate semantic version in renovate/k8s/cronjob.yaml"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
version="${BASH_REMATCH[1]}"
|
|
||||||
echo "using Renovate $version"
|
|
||||||
printf 'version=%s\n' "$version" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: Prepare pinned validation tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh kubeconform node)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
|
|
||||||
- name: Validate Renovate repository config
|
|
||||||
shell: bash
|
|
||||||
env:
|
|
||||||
RENOVATE_VERSION: ${{ steps.image.outputs.version }}
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
npm_cache="$(mktemp -d "${RUNNER_TEMP:-/tmp}/renovate-npm-cache.XXXXXXXX")"
|
|
||||||
trap 'rm -rf "$npm_cache"' EXIT
|
|
||||||
NPM_CONFIG_CACHE="$npm_cache" RENOVATE_CONFIG_FILE="$PWD/renovate/renovate.json" \
|
|
||||||
npm exec --yes --package="renovate@${RENOVATE_VERSION}" -- renovate-config-validator
|
|
||||||
|
|
||||||
# The CronJob cannot read the repository, so renovate/k8s/configmap.yaml
|
|
||||||
# carries an inlined copy of the config. Fail if it no longer matches.
|
|
||||||
- name: Check the generated Renovate ConfigMap
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
./.gitea/workflows/sync-renovate-configmap.sh --check
|
|
||||||
|
|
||||||
- name: Validate Renovate Kubernetes manifests
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
kubeconform \
|
|
||||||
-strict \
|
|
||||||
-ignore-missing-schemas \
|
|
||||||
-summary \
|
|
||||||
renovate/k8s/namespace.yaml \
|
|
||||||
renovate/k8s/configmap.yaml \
|
|
||||||
renovate/k8s/cronjob.yaml
|
|
||||||
|
|
||||||
- name: Validate Renovate Compose file
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
source .gitea/workflows/compose-lint.sh
|
|
||||||
mapfile -t safe_flags < <(compose_safe_flags)
|
|
||||||
validate_compose_file renovate/renovate-compose.yaml \
|
|
||||||
${safe_flags[@]+"${safe_flags[@]}"}
|
|
||||||
@@ -1,98 +0,0 @@
|
|||||||
name: renovate-run
|
|
||||||
|
|
||||||
on:
|
|
||||||
workflow_dispatch:
|
|
||||||
inputs:
|
|
||||||
repositories:
|
|
||||||
description: "Repositories to scan (comma-separated)"
|
|
||||||
required: false
|
|
||||||
default: "forust/homelab"
|
|
||||||
log_level:
|
|
||||||
description: "Renovate log level"
|
|
||||||
required: false
|
|
||||||
default: "info"
|
|
||||||
type: choice
|
|
||||||
options:
|
|
||||||
- info
|
|
||||||
- debug
|
|
||||||
dry_run:
|
|
||||||
description: "Plan only, do not open or update PRs"
|
|
||||||
required: false
|
|
||||||
default: false
|
|
||||||
type: boolean
|
|
||||||
|
|
||||||
# Renovate writes through its own bot PAT, passed in as RENOVATE_TOKEN, so the
|
|
||||||
# Actions token is only ever used to read the checkout.
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
|
|
||||||
concurrency:
|
|
||||||
group: renovate-run
|
|
||||||
cancel-in-progress: false
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
run-renovate:
|
|
||||||
if: github.ref == 'refs/heads/main'
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 60
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
|
||||||
with:
|
|
||||||
ref: refs/heads/main
|
|
||||||
|
|
||||||
# renovate/k8s/cronjob.yaml is the single source of truth for the image tag.
|
|
||||||
# Reading it here means this workflow validates and runs the exact version
|
|
||||||
# that is deployed, instead of a copy that silently goes stale.
|
|
||||||
- name: Resolve the deployed Renovate image
|
|
||||||
id: image
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
image="$(sed -n 's|.*image:[[:space:]]*\(renovate/renovate:[^[:space:]]*\).*|\1|p' \
|
|
||||||
renovate/k8s/cronjob.yaml | head -1)"
|
|
||||||
if [[ ! "$image" =~ ^renovate/renovate:[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
|
||||||
echo "::error::expected a pinned renovate/renovate semantic version in renovate/k8s/cronjob.yaml"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
echo "using $image"
|
|
||||||
printf 'image=%s\n' "$image" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: Validate Renovate config
|
|
||||||
shell: bash
|
|
||||||
env:
|
|
||||||
RENOVATE_IMAGE: ${{ steps.image.outputs.image }}
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
docker run --rm \
|
|
||||||
-v "$PWD/renovate/renovate.json:/opt/renovate/renovate.json:ro" \
|
|
||||||
-e RENOVATE_CONFIG_FILE=/opt/renovate/renovate.json \
|
|
||||||
"$RENOVATE_IMAGE" \
|
|
||||||
renovate-config-validator
|
|
||||||
|
|
||||||
- name: Run Renovate
|
|
||||||
shell: bash
|
|
||||||
env:
|
|
||||||
RENOVATE_TOKEN: ${{ secrets.RENOVATE_TOKEN }}
|
|
||||||
RENOVATE_GITHUB_COM_TOKEN: ${{ secrets.RENOVATE_GITHUB_COM_TOKEN }}
|
|
||||||
RENOVATE_REPOSITORIES: ${{ inputs.repositories }}
|
|
||||||
RENOVATE_DRY_RUN: ${{ inputs.dry_run && 'full' || '' }}
|
|
||||||
LOG_LEVEL: ${{ inputs.log_level }}
|
|
||||||
RENOVATE_IMAGE: ${{ steps.image.outputs.image }}
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
: "${RENOVATE_TOKEN:?missing RENOVATE_TOKEN secret — add a renovate-bot PAT in repo/org Actions secrets}"
|
|
||||||
|
|
||||||
docker run --rm \
|
|
||||||
-v "$PWD/renovate/renovate.json:/opt/renovate/renovate.json:ro" \
|
|
||||||
-e RENOVATE_PLATFORM=gitea \
|
|
||||||
-e RENOVATE_ENDPOINT=https://git.forust.xyz/api/v1 \
|
|
||||||
-e RENOVATE_TOKEN="$RENOVATE_TOKEN" \
|
|
||||||
-e RENOVATE_GITHUB_COM_TOKEN="${RENOVATE_GITHUB_COM_TOKEN:-}" \
|
|
||||||
-e RENOVATE_REPOSITORIES="${RENOVATE_REPOSITORIES:-forust/homelab}" \
|
|
||||||
-e RENOVATE_DRY_RUN="${RENOVATE_DRY_RUN:-}" \
|
|
||||||
-e RENOVATE_CONFIG_FILE=/opt/renovate/renovate.json \
|
|
||||||
-e RENOVATE_BASE_DIR=/tmp/renovate \
|
|
||||||
-e LOG_LEVEL="${LOG_LEVEL:-info}" \
|
|
||||||
"$RENOVATE_IMAGE"
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
# kubectl emits a List for files containing multiple resources.
|
|
||||||
(if .kind == "List" then .items[] else . end)
|
|
||||||
| (.metadata.namespace // "default") as $ns
|
|
||||||
| [
|
|
||||||
(.. | objects
|
|
||||||
| (.secretRef? // empty), (.secretKeyRef? // empty), (.secret? // empty)
|
|
||||||
| select(.optional != true)
|
|
||||||
| .name // .secretName // empty),
|
|
||||||
(.. | objects | .imagePullSecrets[]?.name)
|
|
||||||
]
|
|
||||||
| unique[]
|
|
||||||
| select(. != null and . != "")
|
|
||||||
| "\($ns) \(.)"
|
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# The SSH client submits once and follows durable stages on workstation.
|
|
||||||
set -euo pipefail
|
|
||||||
: "${DEPLOY_HOST:?missing DEPLOY_HOST}"
|
|
||||||
: "${DEPLOY_USER:?missing DEPLOY_USER}"
|
|
||||||
: "${DEPLOY_KEY:?missing DEPLOY_SSH_KEY}"
|
|
||||||
: "${DEPLOY_KNOWN_HOSTS:?configure pinned DEPLOY_KNOWN_HOSTS}"
|
|
||||||
: "${DEPLOY_RUN_ID:?missing DEPLOY_RUN_ID}"
|
|
||||||
[[ "$DEPLOY_USER" =~ ^[A-Za-z_][A-Za-z0-9_.-]*$ ]] || exit 1
|
|
||||||
[[ "$DEPLOY_HOST" =~ ^[A-Za-z0-9_.:-]+$ ]] || exit 1
|
|
||||||
[[ "$DEPLOY_RUN_ID" =~ ^[0-9]+-[0-9]+$ ]] || exit 1
|
|
||||||
[[ "${DEPLOY_PORT:-22}" =~ ^[0-9]+$ ]] || exit 1
|
|
||||||
key_dir="$(mktemp -d "${RUNNER_TEMP:-/tmp}/homelab-deploy-key.XXXXXXXX")"
|
|
||||||
trap 'rm -rf "$key_dir"' EXIT
|
|
||||||
chmod 700 "$key_dir"
|
|
||||||
printf '%s\n' "$DEPLOY_KEY" >"$key_dir/key"
|
|
||||||
printf '%s\n' "$DEPLOY_KNOWN_HOSTS" >"$key_dir/known_hosts"
|
|
||||||
chmod 600 "$key_dir/key" "$key_dir/known_hosts"
|
|
||||||
ssh_opts=(-i "$key_dir/key" -p "${DEPLOY_PORT:-22}" -o BatchMode=yes -o StrictHostKeyChecking=yes
|
|
||||||
-o "UserKnownHostsFile=$key_dir/known_hosts" -o ConnectTimeout=15
|
|
||||||
-o ServerAliveInterval=15 -o ServerAliveCountMax=4)
|
|
||||||
controller=.local/lib/homelab-deploy/controller.py
|
|
||||||
case "${1:?start, apply, verify, smoke or summary required}" in
|
|
||||||
start)
|
|
||||||
python3 - <<'PY' >"$key_dir/request.json"
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
release = json.loads(Path('release.json').read_text())
|
|
||||||
print(json.dumps({'release': release, 'mode': os.environ.get('DEPLOY_MODE', 'changed'),
|
|
||||||
'refresh_images': os.environ.get('REFRESH_IMAGES', 'false') == 'true'}))
|
|
||||||
PY
|
|
||||||
for attempt in 1 2 3; do
|
|
||||||
rc=0
|
|
||||||
# shellcheck disable=SC2029 # The run ID and operation are validated local arguments, not remote variables.
|
|
||||||
ssh "${ssh_opts[@]}" "$DEPLOY_USER@$DEPLOY_HOST" python3 "$controller" start "$DEPLOY_RUN_ID" <"$key_dir/request.json" || rc=$?
|
|
||||||
[ "$rc" -eq 0 ] && exit 0
|
|
||||||
[ "$rc" -eq 255 ] || exit "$rc"
|
|
||||||
sleep 5
|
|
||||||
done
|
|
||||||
exit "$rc"
|
|
||||||
;;
|
|
||||||
apply|verify|smoke)
|
|
||||||
result=0
|
|
||||||
for attempt in 1 2 3; do
|
|
||||||
rc=0
|
|
||||||
# shellcheck disable=SC2029 # The run ID and operation are validated local arguments, not remote variables.
|
|
||||||
ssh "${ssh_opts[@]}" "$DEPLOY_USER@$DEPLOY_HOST" python3 "$controller" follow "$DEPLOY_RUN_ID" "$1" || rc=$?
|
|
||||||
[ "$rc" -eq 0 ] && break
|
|
||||||
[ "$rc" -eq 255 ] || { result="$rc"; break; }
|
|
||||||
echo "SSH disconnected; reconnecting to the existing deploy ($attempt/3)"
|
|
||||||
if [ "$attempt" -eq 3 ]; then result=255; break; fi
|
|
||||||
sleep 5
|
|
||||||
done
|
|
||||||
exit "$result"
|
|
||||||
;;
|
|
||||||
summary)
|
|
||||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
rc=0
|
|
||||||
# shellcheck disable=SC2029 # The run ID is validated above.
|
|
||||||
ssh "${ssh_opts[@]}" "$DEPLOY_USER@$DEPLOY_HOST" python3 "$controller" summary "$DEPLOY_RUN_ID" >"$key_dir/deploy-summary.md" || rc=$?
|
|
||||||
if [ "$rc" -eq 0 ]; then
|
|
||||||
cat "$key_dir/deploy-summary.md" >>"$GITHUB_STEP_SUMMARY" || echo "WARNING: cannot write the deploy summary"
|
|
||||||
else
|
|
||||||
echo 'Deploy summary is unavailable. The SSH connection failed or the controller did not respond. Check the job log.' >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
;;
|
|
||||||
*) echo "Unknown SSH operation: $1" >&2; exit 1 ;;
|
|
||||||
esac
|
|
||||||
@@ -1,55 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Regenerates renovate/k8s/configmap.yaml from renovate/renovate.json.
|
|
||||||
#
|
|
||||||
# renovate/renovate.json is the single source of truth: the CronJob, the Compose
|
|
||||||
# file and the renovate-run workflow all mount that exact file. A ConfigMap cannot
|
|
||||||
# read a file from the repository, so the same bytes are inlined here as a literal
|
|
||||||
# block. This script keeps the copy honest:
|
|
||||||
#
|
|
||||||
# .gitea/workflows/sync-renovate-configmap.sh # rewrite in place
|
|
||||||
# .gitea/workflows/sync-renovate-configmap.sh --check # fail if out of date
|
|
||||||
#
|
|
||||||
# renovate-ci runs the --check form on every PR and push, so a config change that
|
|
||||||
# forgets to regenerate the ConfigMap cannot be merged.
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
repo="$(git -C "$here" rev-parse --show-toplevel)"
|
|
||||||
|
|
||||||
src="$repo/renovate/renovate.json"
|
|
||||||
dst="$repo/renovate/k8s/configmap.yaml"
|
|
||||||
[ -f "$src" ] || {
|
|
||||||
echo "missing $src" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
render() {
|
|
||||||
cat <<'HEADER'
|
|
||||||
# GENERATED FILE - do not edit by hand.
|
|
||||||
# Source: renovate/renovate.json
|
|
||||||
# Regenerate: .gitea/workflows/sync-renovate-configmap.sh
|
|
||||||
# Verify: .gitea/workflows/sync-renovate-configmap.sh --check
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: renovate-config
|
|
||||||
namespace: renovate
|
|
||||||
data:
|
|
||||||
renovate.json: |
|
|
||||||
HEADER
|
|
||||||
sed 's/^/ /' "$src"
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ "${1:-}" = "--check" ]; then
|
|
||||||
if ! diff -u "$dst" <(render) >/dev/null 2>&1; then
|
|
||||||
echo "ERROR: $dst is out of sync with renovate/renovate.json"
|
|
||||||
echo "Run: .gitea/workflows/sync-renovate-configmap.sh"
|
|
||||||
diff -u "$dst" <(render) || true
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
echo "renovate/k8s/configmap.yaml is in sync with renovate/renovate.json"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
render >"$dst"
|
|
||||||
echo "wrote $dst"
|
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
# Pinned versions of the CI tools installed by install-ci-tools.sh.
|
|
||||||
# Renovate keeps these up to date (see customManagers in renovate/renovate.json).
|
|
||||||
#
|
|
||||||
# Every version here except NODE_VERSION matches what was already installed on
|
|
||||||
# the runner, so pinning them changes what CI does not at all. It changes what
|
|
||||||
# CI does when the runner is rebuilt with something else: today
|
|
||||||
# install-ci-tools.sh finds the pinned version already on PATH and installs
|
|
||||||
# nothing, and a runner that drifts gets the pinned one installed over it.
|
|
||||||
#
|
|
||||||
# The renovate image version is NOT pinned here: renovate/k8s/cronjob.yaml is the
|
|
||||||
# single source of truth and the workflows read the tag from it, so there is
|
|
||||||
# nothing to drift.
|
|
||||||
ACTIONLINT_VERSION="1.7.7"
|
|
||||||
SHELLCHECK_VERSION="0.11.0"
|
|
||||||
KUBECONFORM_VERSION="0.8.0"
|
|
||||||
PRETTIER_VERSION="3.8.1"
|
|
||||||
RUFF_VERSION="0.16.8"
|
|
||||||
YAMLLINT_VERSION="1.38.0"
|
|
||||||
HADOLINT_VERSION="2.14.0"
|
|
||||||
# pip-audit reads the advisory database over the network, so a floating version
|
|
||||||
# would make the same commit report different things on different days. Pin it
|
|
||||||
# like the rest: the advisories themselves are the moving part, not the tool.
|
|
||||||
PIP_AUDIT_VERSION="2.10.1"
|
|
||||||
# uv builds the throwaway venv the pytest job runs in, and unpacks the PyPI
|
|
||||||
# wheels for ruff, yamllint and pip-audit.
|
|
||||||
UV_VERSION="0.12.17"
|
|
||||||
# node runs `npm ci` for the frontend tests and the npm audit, and it is the one
|
|
||||||
# pin here that does NOT come from the runner: the runner's system node is a
|
|
||||||
# rolling Arch package (it was node 26 with no npm at all when this was pinned),
|
|
||||||
# and the panel image is node:22-alpine. Pinned to the image's major on purpose,
|
|
||||||
# so the tree that gets tested is the tree that gets built. Renovate keeps this
|
|
||||||
# in step with the Dockerfile's node: tag via the "node runtime" group.
|
|
||||||
NODE_VERSION="22.23.3"
|
|
||||||
|
|
||||||
# Secret-reference regression tests parse rendered Kubernetes objects.
|
|
||||||
JQ_VERSION="1.8.1"
|
|
||||||
|
|
||||||
# BuildKit is the only auxiliary CI container; jobs themselves stay on the host.
|
|
||||||
BUILDKIT_IMAGE="moby/buildkit:v0.33.1"
|
|
||||||
+13
-47
@@ -2,15 +2,19 @@
|
|||||||
sync.ffs_lock
|
sync.ffs_lock
|
||||||
.sync.ffs_db
|
.sync.ffs_db
|
||||||
|
|
||||||
# Copyparty
|
# Environment
|
||||||
*.hist/
|
.env
|
||||||
|
.env.anna
|
||||||
|
.env.forust
|
||||||
|
.env.*
|
||||||
|
!.env.*example
|
||||||
|
|
||||||
# Volumes, configs and data directories
|
# Volumes and data directories
|
||||||
gitea/gitea-db/
|
gitea/gitea-db/
|
||||||
gitea/gitea-data/*
|
gitea/gitea-data/*
|
||||||
n8n/n8n-data/*
|
n8n/n8n-data/*
|
||||||
n8n/n8n-node-data/*
|
n8n/n8n-node-data/*
|
||||||
adguardhome/conf/*
|
adguardhome/data/*
|
||||||
dockmon/data/*
|
dockmon/data/*
|
||||||
portainer/portainer_data/*
|
portainer/portainer_data/*
|
||||||
metube/MeTube_downloads
|
metube/MeTube_downloads
|
||||||
@@ -18,13 +22,6 @@ uptime-kuma/data/
|
|||||||
termix/termix-data/*
|
termix/termix-data/*
|
||||||
cfddns/config.json
|
cfddns/config.json
|
||||||
checkmk/checkmk/*
|
checkmk/checkmk/*
|
||||||
downtify/Downtify_downloads
|
|
||||||
headscale/config/*
|
|
||||||
headscale/data/*
|
|
||||||
# NetBird local hostnames and generated secrets
|
|
||||||
netbird/.env
|
|
||||||
netbird/secrets/
|
|
||||||
searxng/core-config/*
|
|
||||||
|
|
||||||
# Steaming services files
|
# Steaming services files
|
||||||
streaming/jellyfin/*
|
streaming/jellyfin/*
|
||||||
@@ -35,28 +32,19 @@ streaming/data/*
|
|||||||
streaming/qbittorrent/*
|
streaming/qbittorrent/*
|
||||||
streaming/prowlarr/*
|
streaming/prowlarr/*
|
||||||
|
|
||||||
# Homepage
|
# Homepage
|
||||||
homepages/forust_files/.well-known/*
|
homepages/forust_files/assets/images/team/*
|
||||||
|
|
||||||
|
|
||||||
# Traefik files
|
# Traefik files
|
||||||
traefik/letsencrypt/acme.json
|
traefik/letsencrypt/acme.json
|
||||||
traefik/dynamic/fileservers.yml
|
|
||||||
traefik/dynamic/*.local.y*ml.*
|
|
||||||
traefik/dynamic/*.external.y*ml
|
|
||||||
traefik/k8s/fileservers.y*ml
|
|
||||||
traefik/k8s/aliasHeadersStrategy.md
|
|
||||||
|
|
||||||
traefik/logs/*
|
traefik/logs/*
|
||||||
|
|
||||||
# SSL Certificates
|
|
||||||
adguardhome/certs/*
|
|
||||||
traefik/certs/*
|
traefik/certs/*
|
||||||
certs/
|
|
||||||
|
|
||||||
# Monitoring
|
# Monitoring
|
||||||
monitoring/prometheus.yml
|
monitoring/prometheus.yml
|
||||||
|
|
||||||
# Python
|
# Python
|
||||||
.python-version
|
.python-version
|
||||||
venv/
|
venv/
|
||||||
pyc
|
pyc
|
||||||
@@ -86,32 +74,10 @@ replacements.txt
|
|||||||
|
|
||||||
# Git
|
# Git
|
||||||
.gitattributes
|
.gitattributes
|
||||||
# Gitea/github Runners
|
|
||||||
.runner
|
|
||||||
|
|
||||||
# Misc
|
# Misc
|
||||||
.DS_Store
|
.DS_Store
|
||||||
.idea
|
.idea
|
||||||
|
|
||||||
# Temp files
|
# Temp files
|
||||||
temp/*
|
edu_master/temp/
|
||||||
# Local-only tooling scratch space (pinned CI tools, verification scripts)
|
|
||||||
tmp/
|
|
||||||
|
|
||||||
# Environment
|
|
||||||
.env
|
|
||||||
.env.anna
|
|
||||||
.env.forust
|
|
||||||
.env.*
|
|
||||||
!*example
|
|
||||||
|
|
||||||
# kubernetes
|
|
||||||
*/k8s/*secret*
|
|
||||||
!*/k8s/*secret*.example
|
|
||||||
**/k8s/*secret*
|
|
||||||
!**/k8s/*secret*.example
|
|
||||||
# Local-only tweaks, not for upstream
|
|
||||||
prometheus-stack/k8s/grafana-values.yaml
|
|
||||||
traefik/k8s/local-tls.yaml
|
|
||||||
converters/k8s/config.yaml
|
|
||||||
convertx/k8s/config.yaml
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
ignored:
|
|
||||||
- DL3008
|
|
||||||
- DL3042
|
|
||||||
- DL3018
|
|
||||||
- DL3059
|
|
||||||
trustedRegistries:
|
|
||||||
- docker.io
|
|
||||||
- ghcr.io
|
|
||||||
- quay.io
|
|
||||||
- gcr.forust.xyz
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
{
|
|
||||||
"default": true,
|
|
||||||
"MD013": false,
|
|
||||||
"MD024": false,
|
|
||||||
"MD033": false,
|
|
||||||
"MD041": false,
|
|
||||||
"MD046": false
|
|
||||||
}
|
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
bracketSameLine: true
|
|
||||||
htmlWhitespaceSensitivity: css
|
|
||||||
printWidth: 120
|
|
||||||
tabWidth: 2
|
|
||||||
trailingComma: all
|
|
||||||
proseWrap: preserve
|
|
||||||
endOfLine: lf
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
extends: default
|
|
||||||
|
|
||||||
rules:
|
|
||||||
comments:
|
|
||||||
min-spaces-from-content: 1
|
|
||||||
comments-indentation: false
|
|
||||||
document-start: disable
|
|
||||||
line-length: disable
|
|
||||||
braces:
|
|
||||||
min-spaces-inside: 0
|
|
||||||
max-spaces-inside: 1
|
|
||||||
brackets:
|
|
||||||
min-spaces-inside: 0
|
|
||||||
max-spaces-inside: 1
|
|
||||||
indentation:
|
|
||||||
spaces: 2
|
|
||||||
indent-sequences: consistent
|
|
||||||
truthy:
|
|
||||||
allowed-values:
|
|
||||||
- "true"
|
|
||||||
- "false"
|
|
||||||
- "on"
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
{
|
|
||||||
"tab_size": 2,
|
|
||||||
"soft_wrap": "prefer_line",
|
|
||||||
"preferred_line_length": 120,
|
|
||||||
"format_on_save": "on",
|
|
||||||
"languages": {
|
|
||||||
"YAML": {
|
|
||||||
"tab_size": 2,
|
|
||||||
"hard_tabs": false,
|
|
||||||
"format_on_save": "on",
|
|
||||||
"formatter": {
|
|
||||||
"language_server": { "name": "yaml-language-server" },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"Python": {
|
|
||||||
"tab_size": 4,
|
|
||||||
"format_on_save": "on",
|
|
||||||
"language_servers": ["pyright", "ruff"],
|
|
||||||
"formatter": {
|
|
||||||
"language_server": { "name": "ruff" },
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"lsp": {
|
|
||||||
"yaml-language-server": {
|
|
||||||
"settings": {
|
|
||||||
"yaml": {
|
|
||||||
"schemas": {
|
|
||||||
"kubernetes": ["**/k8s/*.yaml", "**/k8s/*.yml"],
|
|
||||||
},
|
|
||||||
"validate": true,
|
|
||||||
"completion": true,
|
|
||||||
"format": {
|
|
||||||
"enable": true,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
+23
-21
@@ -1,48 +1,50 @@
|
|||||||
services:
|
services:
|
||||||
adguard:
|
adguard:
|
||||||
image: adguard/adguardhome:v0.107.79
|
image: adguard/adguardhome:latest
|
||||||
container_name: adguardhome
|
container_name: adguardhome
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- TZ=${TZ}
|
||||||
ports:
|
ports:
|
||||||
- "53:53/tcp"
|
- "53:53/tcp"
|
||||||
- "53:53/udp"
|
- "53:53/udp"
|
||||||
- "853:853/tcp" # DNS over TLS
|
|
||||||
# - "67:67/udp" # DHCP
|
# - "67:67/udp" # DHCP
|
||||||
# - "68:68/tcp" # DHCP
|
# - "68:68/tcp" # DHCP
|
||||||
# - "3000:3000/tcp"
|
- "3000:3000/tcp"
|
||||||
volumes:
|
volumes:
|
||||||
- data:/opt/adguardhome/work
|
- ./data/work:/opt/adguardhome/work
|
||||||
- ./conf:/opt/adguardhome/conf
|
- ./data/conf:/opt/adguardhome/conf
|
||||||
- ./certs:/certs:ro
|
networks:
|
||||||
|
- traefik-proxy
|
||||||
labels:
|
labels:
|
||||||
- "traefik.enable=true"
|
- "traefik.enable=true"
|
||||||
- "traefik.http.services.adguard.loadbalancer.server.port=3000"
|
- "traefik.docker.network=traefik-proxy"
|
||||||
|
|
||||||
# Prod Router
|
# Prod Router
|
||||||
- "traefik.http.routers.adguard.rule=Host(`dns.forust.xyz`) || Host(`adguard.forust.xyz`)"
|
- "traefik.http.routers.adguard.rule=Host(`adguard.forust.xyz`)"
|
||||||
- "traefik.http.routers.adguard.entrypoints=websecure"
|
- "traefik.http.routers.adguard.entrypoints=websecure"
|
||||||
- "traefik.http.routers.adguard.tls.certresolver=letsencrypt"
|
- "traefik.http.routers.adguard.middlewares=security-headers@file"
|
||||||
|
- "traefik.http.routers.adguard.service=adguard"
|
||||||
|
- "traefik.http.routers.adguard.tls=true"
|
||||||
|
- "traefik.http.services.adguard.loadbalancer.server.port=3000"
|
||||||
|
|
||||||
# Local Router
|
# Local Router
|
||||||
- "traefik.http.routers.adguard-local.rule=Host(`adguard.workstation.internal`) || Host(`dns.workstation.internal`)"
|
- "traefik.http.routers.adguard-local.rule=Host(`adguard.workstation.internal`) || Host(`adguard.internal`)"
|
||||||
- "traefik.http.routers.adguard-local.entrypoints=websecure"
|
- "traefik.http.routers.adguard-local.entrypoints=websecure"
|
||||||
|
- "traefik.http.routers.adguard-local.middlewares=security-headers@file"
|
||||||
|
- "traefik.http.routers.adguard-local.service=adguard"
|
||||||
- "traefik.http.routers.adguard-local.tls=true"
|
- "traefik.http.routers.adguard-local.tls=true"
|
||||||
|
|
||||||
# Dev Router
|
# Dev Router
|
||||||
- "traefik.http.routers.adguard-dev.rule=Host(`adguard.gigaforust.internal`) || Host(`dns.gigaforust.internal`)"
|
- "traefik.http.routers.adguard-dev.rule=Host(`adguard.gigaforust.internal`)"
|
||||||
- "traefik.http.routers.adguard-dev.entrypoints=websecure"
|
- "traefik.http.routers.adguard-dev.entrypoints=websecure"
|
||||||
|
- "traefik.http.routers.adguard-dev.middlewares=security-headers@file"
|
||||||
|
- "traefik.http.routers.adguard-dev.service=adguard"
|
||||||
- "traefik.http.routers.adguard-dev.tls=true"
|
- "traefik.http.routers.adguard-dev.tls=true"
|
||||||
# DoH Router
|
|
||||||
- "traefik.http.routers.dns-over-https.rule=(Host(`dns.forust.xyz`) || Host(`adguard.forust.xyz`)) && PathPrefix(`/dns-query`)"
|
|
||||||
- "traefik.http.routers.dns-over-https.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.dns-over-https.tls.certresolver=letsencrypt"
|
|
||||||
|
|
||||||
# Glance Metadata
|
|
||||||
- glance.name=adguard
|
- glance.name=adguard
|
||||||
- glance.url=https://adguard.forust.xyz/
|
- glance.url=https://adguard.forust.xyz/
|
||||||
- glance.description=AdGuard Home is a network-wide software for blocking ads.
|
- glance.description=AdGuard Home is a network-wide software for blocking ads.
|
||||||
networks:
|
|
||||||
- proxy
|
|
||||||
volumes:
|
|
||||||
data:
|
|
||||||
networks:
|
networks:
|
||||||
proxy:
|
traefik-proxy:
|
||||||
external: true
|
external: true
|
||||||
@@ -1,127 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: adguard-lb-service
|
|
||||||
namespace: adguard
|
|
||||||
annotations:
|
|
||||||
metallb.io/loadBalancerIPs: "192.168.80.3"
|
|
||||||
spec:
|
|
||||||
type: LoadBalancer
|
|
||||||
externalTrafficPolicy: Local
|
|
||||||
selector:
|
|
||||||
app: adguard
|
|
||||||
ports:
|
|
||||||
- name: dns-udp
|
|
||||||
port: 53
|
|
||||||
targetPort: 53
|
|
||||||
protocol: UDP
|
|
||||||
- name: dns-tcp
|
|
||||||
port: 53
|
|
||||||
targetPort: 53
|
|
||||||
protocol: TCP
|
|
||||||
- name: dot
|
|
||||||
port: 853
|
|
||||||
targetPort: 853
|
|
||||||
protocol: TCP
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: adguard-service
|
|
||||||
namespace: adguard
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: adguard
|
|
||||||
ports:
|
|
||||||
- port: 3000
|
|
||||||
name: webui
|
|
||||||
targetPort: 3000
|
|
||||||
- port: 53
|
|
||||||
name: dns
|
|
||||||
targetPort: 53
|
|
||||||
protocol: UDP
|
|
||||||
- port: 53
|
|
||||||
name: dns-tcp
|
|
||||||
targetPort: 53
|
|
||||||
protocol: TCP
|
|
||||||
- port: 853
|
|
||||||
name: dot
|
|
||||||
targetPort: 853
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
reloader.stakater.com/auto: "true"
|
|
||||||
name: adguard-deployment
|
|
||||||
namespace: adguard
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: adguard
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: adguard
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: adguard
|
|
||||||
image: adguard/adguardhome:v0.107.79
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
memory: "1.5Gi"
|
|
||||||
cpu: "300m"
|
|
||||||
requests:
|
|
||||||
memory: "512Mi"
|
|
||||||
cpu: "50m"
|
|
||||||
ports:
|
|
||||||
- containerPort: 3000
|
|
||||||
name: webui
|
|
||||||
- containerPort: 53
|
|
||||||
name: dns
|
|
||||||
- containerPort: 853
|
|
||||||
name: dot
|
|
||||||
readinessProbe:
|
|
||||||
tcpSocket:
|
|
||||||
port: dns
|
|
||||||
initialDelaySeconds: 5
|
|
||||||
periodSeconds: 5
|
|
||||||
successThreshold: 1
|
|
||||||
failureThreshold: 3
|
|
||||||
volumeMounts:
|
|
||||||
- name: adguard-data
|
|
||||||
mountPath: /opt/adguardhome/work
|
|
||||||
subPath: work
|
|
||||||
- name: adguard-data
|
|
||||||
mountPath: /opt/adguardhome/conf
|
|
||||||
subPath: conf
|
|
||||||
- name: adguard-certs
|
|
||||||
mountPath: /certs
|
|
||||||
readOnly: true
|
|
||||||
volumes:
|
|
||||||
- name: adguard-data
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: adguard-pvc
|
|
||||||
- name: adguard-certs
|
|
||||||
secret:
|
|
||||||
secretName: adguard-certs
|
|
||||||
items:
|
|
||||||
- key: tls.crt
|
|
||||||
path: fullchain.pem
|
|
||||||
- key: tls.key
|
|
||||||
path: privkey.pem
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: adguard-pvc
|
|
||||||
namespace: adguard
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 2Gi
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: adguard-certs
|
|
||||||
namespace: adguard
|
|
||||||
spec:
|
|
||||||
secretName: adguard-certs
|
|
||||||
dnsNames:
|
|
||||||
- dns.forust.xyz
|
|
||||||
issuerRef:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
kind: ClusterIssuer
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: adguard-prod
|
|
||||||
namespace: adguard
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`dns.forust.xyz`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: adguard-service
|
|
||||||
port: 3000
|
|
||||||
- match: (Host(`dns.forust.xyz`)) && PathPrefix(`/dns-query`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: adguard-service
|
|
||||||
port: 3000
|
|
||||||
tls:
|
|
||||||
secretName: adguard-certs
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: adguard-local
|
|
||||||
namespace: adguard
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`adguard.workstation.internal`) || Host(`dns.workstation.internal`) || Host(`adguard.gigaforust.internal`) || Host(`dns.gigaforust.internal`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: adguard-service
|
|
||||||
port: 3000
|
|
||||||
- match: (Host(`adguard.workstation.internal`) || Host(`dns.workstation.internal`) || Host(`adguard.gigaforust.internal`) || Host(`dns.gigaforust.internal`)) && PathPrefix(`/dns-query`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: adguard-service
|
|
||||||
port: 3000
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: internal-wildcard-tls
|
|
||||||
namespace: adguard
|
|
||||||
spec:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
dnsNames:
|
|
||||||
- "*.workstation.internal"
|
|
||||||
- "*.gigaforust.internal"
|
|
||||||
- workstation.internal
|
|
||||||
- gigaforust.internal
|
|
||||||
issuerRef:
|
|
||||||
name: internal-ca
|
|
||||||
kind: ClusterIssuer
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: adguard
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
kubectl apply -f k8s/namespace.yaml && \
|
|
||||||
kubectl create secret tls adguard-certs -n adguard \
|
|
||||||
--cert=certs/fullchain.pem \
|
|
||||||
--key=certs/privkey.pem --dry-run=client -o yaml > \
|
|
||||||
k8s/secrets.yaml
|
|
||||||
|
|
||||||
# OR WITH NO FILE CREATION:
|
|
||||||
kubectl create secret tls adguard-certs -n adguard \
|
|
||||||
--cert=certs/fullchain.pem --key=certs/privkey.pem \
|
|
||||||
--save-config
|
|
||||||
+22
-11
@@ -1,6 +1,6 @@
|
|||||||
services:
|
services:
|
||||||
postgresql:
|
postgresql:
|
||||||
image: docker.io/library/postgres:15.19-alpine
|
image: docker.io/library/postgres:15-alpine
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
env_file:
|
env_file:
|
||||||
- .env
|
- .env
|
||||||
@@ -26,9 +26,9 @@ services:
|
|||||||
command: server
|
command: server
|
||||||
container_name: authentik-server
|
container_name: authentik-server
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
# ports:
|
ports:
|
||||||
# - ${PORT_HTTP:-9000}:9000
|
- ${PORT_HTTP:-9000}:9000
|
||||||
# - ${PORT_HTTPS:-9443}:9443
|
- ${PORT_HTTPS:-9443}:9443
|
||||||
env_file:
|
env_file:
|
||||||
- .env
|
- .env
|
||||||
environment:
|
environment:
|
||||||
@@ -37,28 +37,39 @@ services:
|
|||||||
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
|
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
|
||||||
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
|
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
|
||||||
AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY:?secret key required}
|
AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY:?secret key required}
|
||||||
volumes:
|
|
||||||
- ./media:/media
|
|
||||||
- ./custom-templates:/templates
|
|
||||||
labels:
|
labels:
|
||||||
- "traefik.enable=true"
|
- "traefik.enable=true"
|
||||||
|
- "traefik.docker.network=traefik-proxy"
|
||||||
|
# Services
|
||||||
|
# - "traefik.http.services.authentik-server.loadbalancer.server.port=9443"
|
||||||
- "traefik.http.services.authentik-server.loadbalancer.server.port=9000"
|
- "traefik.http.services.authentik-server.loadbalancer.server.port=9000"
|
||||||
|
|
||||||
# Prod Router
|
# Prod Router
|
||||||
- "traefik.http.routers.authentik-server.rule=Host(`auth.forust.xyz`)"
|
- "traefik.http.routers.authentik-server.rule=Host(`auth.forust.xyz`)"
|
||||||
- "traefik.http.routers.authentik-server.entrypoints=websecure"
|
- "traefik.http.routers.authentik-server.entrypoints=websecure"
|
||||||
- "traefik.http.routers.authentik-server.tls.certresolver=letsencrypt"
|
- "traefik.http.routers.authentik-server.middlewares=security-headers@file"
|
||||||
|
- "traefik.http.routers.authentik-server.service=authentik-server"
|
||||||
|
- "traefik.http.routers.authentik-server.tls=true"
|
||||||
|
|
||||||
# Local Router
|
# Local Router
|
||||||
- "traefik.http.routers.authentik-server-local.rule=Host(`auth.workstation.internal`)"
|
- "traefik.http.routers.authentik-server-local.rule=Host(`auth.workstation.internal`) || Host(`auth-dashboard.internal`)"
|
||||||
- "traefik.http.routers.authentik-server-local.entrypoints=websecure"
|
- "traefik.http.routers.authentik-server-local.entrypoints=websecure"
|
||||||
|
- "traefik.http.routers.authentik-server-local.middlewares=security-headers@file"
|
||||||
|
- "traefik.http.routers.authentik-server-local.service=authentik-server"
|
||||||
- "traefik.http.routers.authentik-server-local.tls=true"
|
- "traefik.http.routers.authentik-server-local.tls=true"
|
||||||
|
|
||||||
# Dev Router
|
# Dev Router
|
||||||
- "traefik.http.routers.authentik-server-dev.rule=Host(`auth.gigaforust.internal`)"
|
- "traefik.http.routers.authentik-server-dev.rule=Host(`auth.gigaforust.internal`)"
|
||||||
- "traefik.http.routers.authentik-server-dev.entrypoints=websecure"
|
- "traefik.http.routers.authentik-server-dev.entrypoints=websecure"
|
||||||
- "traefik.http.routers.authentik-server-dev.middlewares=security-headers@file"
|
- "traefik.http.routers.authentik-server-dev.middlewares=security-headers@file"
|
||||||
|
- "traefik.http.routers.authentik-server-dev.service=authentik-server"
|
||||||
- "traefik.http.routers.authentik-server-dev.tls=true"
|
- "traefik.http.routers.authentik-server-dev.tls=true"
|
||||||
|
volumes:
|
||||||
|
- ./media:/media
|
||||||
|
- ./custom-templates:/templates
|
||||||
networks:
|
networks:
|
||||||
- proxy
|
- traefik-proxy
|
||||||
- authentik
|
- authentik
|
||||||
depends_on:
|
depends_on:
|
||||||
postgresql:
|
postgresql:
|
||||||
@@ -91,5 +102,5 @@ volumes:
|
|||||||
driver: local
|
driver: local
|
||||||
networks:
|
networks:
|
||||||
authentik:
|
authentik:
|
||||||
proxy:
|
traefik-proxy:
|
||||||
external: true
|
external: true
|
||||||
@@ -1,101 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: authentik-server-service
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
type: ClusterIP
|
|
||||||
selector:
|
|
||||||
app: authentik-server
|
|
||||||
ports:
|
|
||||||
- port: 9000
|
|
||||||
targetPort: 9000
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: authentik-worker-service
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
type: ClusterIP
|
|
||||||
selector:
|
|
||||||
app: authentik-worker
|
|
||||||
ports:
|
|
||||||
- port: 9000
|
|
||||||
targetPort: 9000
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
reloader.stakater.com/auto: "true"
|
|
||||||
name: authentik-server-deployment
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: authentik-server
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: authentik-server
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: authentik-server
|
|
||||||
image: ghcr.io/goauthentik/server:2026.8.3
|
|
||||||
args: ["server"]
|
|
||||||
envFrom:
|
|
||||||
- configMapRef:
|
|
||||||
name: authentik-config
|
|
||||||
- secretRef:
|
|
||||||
name: authentik-secrets
|
|
||||||
ports:
|
|
||||||
- containerPort: 9000
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "768Mi"
|
|
||||||
cpu: "100m"
|
|
||||||
limits:
|
|
||||||
memory: "1.5Gi"
|
|
||||||
cpu: "1000m"
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
reloader.stakater.com/auto: "true"
|
|
||||||
name: authentik-worker-deployment
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: authentik-worker
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: authentik-worker
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: authentik-worker
|
|
||||||
image: ghcr.io/goauthentik/server:2026.8.3
|
|
||||||
args: ["worker"]
|
|
||||||
securityContext:
|
|
||||||
runAsUser: 0
|
|
||||||
envFrom:
|
|
||||||
- configMapRef:
|
|
||||||
name: authentik-config
|
|
||||||
- secretRef:
|
|
||||||
name: authentik-secrets
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "320Mi"
|
|
||||||
cpu: "100m"
|
|
||||||
limits:
|
|
||||||
memory: "768Mi"
|
|
||||||
cpu: "700m"
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: authentik-prod-tls
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
secretName: authentik-prod-tls
|
|
||||||
dnsNames:
|
|
||||||
- auth.forust.xyz
|
|
||||||
issuerRef:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
kind: ClusterIssuer
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: internal-wildcard-tls
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
dnsNames:
|
|
||||||
- "*.workstation.internal"
|
|
||||||
- "*.gigaforust.internal"
|
|
||||||
- workstation.internal
|
|
||||||
- gigaforust.internal
|
|
||||||
issuerRef:
|
|
||||||
name: internal-ca
|
|
||||||
kind: ClusterIssuer
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: authentik-config
|
|
||||||
namespace: authentik
|
|
||||||
data:
|
|
||||||
AUTHENTIK_IMAGE: ghcr.io/goauthentik/server
|
|
||||||
AUTHENTIK_TAG: "2025.10.2"
|
|
||||||
AUTHENTIK_POSTGRESQL__HOST: postgres.database.svc.cluster.local
|
|
||||||
AUTHENTIK_POSTGRESQL__NAME: authentik
|
|
||||||
AUTHENTIK_ERROR_REPORTING__ENABLED: "true"
|
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: authentik-prod
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`auth.forust.xyz`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: authentik-server-service
|
|
||||||
port: 9000
|
|
||||||
tls:
|
|
||||||
secretName: authentik-prod-tls
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: authentik-local
|
|
||||||
namespace: authentik
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`auth.workstation.internal`) || Host(`auth.gigaforust.internal`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: authentik-server-service
|
|
||||||
port: 9000
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: authentik
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: authentik-secrets
|
|
||||||
namespace: authentik
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
AUTHENTIK_SECRET_KEY: ""
|
|
||||||
AUTHENTIK_POSTGRESQL__PASSWORD: ""
|
|
||||||
AUTHENTIK_POSTGRESQL__USER: authentik
|
|
||||||
AUTHENTIK_BOOTSTRAP_PASSWORD: authentik
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
crds:
|
|
||||||
enabled: true
|
|
||||||
prometheus:
|
|
||||||
servicemonitor:
|
|
||||||
enabled: true
|
|
||||||
interval: 60s
|
|
||||||
scrapeTimeout: 30s
|
|
||||||
labels:
|
|
||||||
release: prometheus-stack
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: ClusterIssuer
|
|
||||||
metadata:
|
|
||||||
name: letsencrypt-staging
|
|
||||||
spec:
|
|
||||||
acme:
|
|
||||||
email: bobrovod@national.shitposting.agency
|
|
||||||
server: https://acme-staging-v02.api.letsencrypt.org/directory
|
|
||||||
privateKeySecretRef:
|
|
||||||
name: letsencrypt-staging-account-key
|
|
||||||
solvers:
|
|
||||||
- http01:
|
|
||||||
ingress:
|
|
||||||
class: traefik
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: ClusterIssuer
|
|
||||||
metadata:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
spec:
|
|
||||||
acme:
|
|
||||||
email: bobrovod@national.shitposting.agency
|
|
||||||
server: https://acme-v02.api.letsencrypt.org/directory
|
|
||||||
privateKeySecretRef:
|
|
||||||
name: letsencrypt-prod-account-key
|
|
||||||
solvers:
|
|
||||||
- http01:
|
|
||||||
ingress:
|
|
||||||
class: traefik
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
-----BEGIN CERTIFICATE-----
|
|
||||||
MIIFFjCCAv6gAwIBAgIUetKpTfEDOn2985FFMu6G26itT+wwDQYJKoZIhvcNAQEN
|
|
||||||
BQAwIzEhMB8GA1UEAxMYaG9tZWxhYiBpbnRlcm5hbCByb290IENBMB4XDTI2MDky
|
|
||||||
MzEyNDA0N1oXDTM2MDkyMDEyNDA0N1owIzEhMB8GA1UEAxMYaG9tZWxhYiBpbnRl
|
|
||||||
cm5hbCByb290IENBMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvmNP
|
|
||||||
ZCOoD8NtNuYJKVXBlTPjX7D7sJCSK5neH7ZbYV5+lmUlEErY8Mik7j37V5k5NfpF
|
|
||||||
Ig85pOjP7RckTPz5V6ek3yaN40s4AL053sN5ZPauDVYjalaEHTgj5sEMqlLACQWI
|
|
||||||
yZmJOZspZykae8dIpQnqCoFpRT4FurJ78v4a0ylnFVLMQn/lyCHedwTjkEdtYWYr
|
|
||||||
ccJy8vQwqkzs/rWvEH1lDqZhennLOrmcCjfonG7D/pruMn4z+6E28p4+ejkRrI6x
|
|
||||||
luak3KnpT1XMeHtgU21hiRGaMDBchHMFgAhnY1qosymKenXvfTZItwgjZbwa1hJI
|
|
||||||
GAiDm+jQDKMjzRZ3rH6Xfc0auUcykNz73PpNu1NGm78nndXwCXcXn1LFKNQJ+r1U
|
|
||||||
sJiyAmUZmXVn4aM4OMf2F38k7wTYIKg7nRGaUkNeKDlNkjA4HvgWw+jwO1KmdHQ/
|
|
||||||
mOem1rosDWHRK01wg+Gga9mQCnhNhxglg3t/UeSic6uOaRsvaz4qkzHq8MbCujVz
|
|
||||||
DpKQjqdikYOAXZOs4KlBLWrS7NaK4NzfSD02pBUErh54ruJfY/bWz9KyXzBD/lQZ
|
|
||||||
VUTKyvUVB0bkVHEdf1jJmX3H4IZRQSF5JPqOBotW6bJI5fEGNBvj9Zxy4nm2WWGz
|
|
||||||
yyP3uWsQz8U/Wdx9nXZLHInTZBsvgLYtKUAWA30CAwEAAaNCMEAwDgYDVR0PAQH/
|
|
||||||
BAQDAgKkMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFEkKm2rxPaK6+O9WD80z
|
|
||||||
BLC6F9QsMA0GCSqGSIb3DQEBDQUAA4ICAQAnFyHz97Umf5VIu+dKTJid7C73VugJ
|
|
||||||
TIar/xJBs/4CxP+znBxhJjXygRoyIfzoVGWcB2ZSL//vL78Qlts79K/Imc9a4RFF
|
|
||||||
wMvCxsRXAEQ4TpeWi3ophPNcs4rhsP+gQKQFtnyKP9519bqpfxp0bTqwOV2o18fn
|
|
||||||
za7rlQViiEnNV58j7CVoM9+mJvVVfBEX1Km+GyJL9GadzbIQ7FxClVJZefCbft93
|
|
||||||
zHVk9gDOw8ys1XGSR2OUCyCLinXO6mqS16CmBb2MAKXq/YyH7E0N8iotAPGtfA8V
|
|
||||||
M/0ddy947rY0xCrtECfWwvGQpJS7NRv/Z9b2jCfXrI5LXmL2nfQRg0y9GE4Vjwr+
|
|
||||||
WxtGU5jOeFt0jQ+xRzcgG0Op+qK3x55l5LSo2hOcOVYbiHxcHEJFgwNi1ADeBFwb
|
|
||||||
q/HdysfURSOghqjIpMMAUabBp+DBUg2EUF7pIaUqbdqExFYcr9EYisEMiNsmKmN+
|
|
||||||
8ZbcOeerFKDQj+t/R0bFXa7UBn2UWsjI8zlR74aa2kLDXwtyz/XlO/FlYm66eBFo
|
|
||||||
2/eYUSeU+S4ej+wUAs/dvjF7f190/DUQGuwOTlLTahqWDztmhCk7qzbECu56CwKT
|
|
||||||
E5Ect2P72UleYwdblkVOVd352AmiwEzdOaziIRrPh8uenEknH6JBYPO3mjk7cCg+
|
|
||||||
GPGcNIBctjdXhg==
|
|
||||||
-----END CERTIFICATE-----
|
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: ClusterIssuer
|
|
||||||
metadata:
|
|
||||||
name: selfsigned
|
|
||||||
spec:
|
|
||||||
selfSigned: {}
|
|
||||||
---
|
|
||||||
# Homelab internal root CA (10y). Install the .crt on clients (see below).
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: internal-ca-root
|
|
||||||
namespace: cert-manager
|
|
||||||
spec:
|
|
||||||
isCA: true
|
|
||||||
commonName: homelab internal root CA
|
|
||||||
duration: 87600h
|
|
||||||
renewBefore: 7200h
|
|
||||||
secretName: internal-ca-root
|
|
||||||
privateKey:
|
|
||||||
algorithm: RSA
|
|
||||||
size: 4096
|
|
||||||
issuerRef:
|
|
||||||
name: selfsigned
|
|
||||||
kind: ClusterIssuer
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: ClusterIssuer
|
|
||||||
metadata:
|
|
||||||
name: internal-ca
|
|
||||||
spec:
|
|
||||||
ca:
|
|
||||||
secretName: internal-ca-root
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: cert-manager
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
CLOUDFLARE_API_TOKEN=YOUR_CLOUDFLARE_API_TOKEN
|
|
||||||
DOMAINS=example.com,dns.example.com,mc.example.com,auth.example.com,ssh.example.com
|
|
||||||
IP4_DOMAINS=
|
|
||||||
IP6_DOMAINS=
|
|
||||||
IP4_PROVIDER=cloudflare.trace
|
|
||||||
IP6_PROVIDER=none # change if you want to update AAAA
|
|
||||||
UPDATE_CRON=@every 5m
|
|
||||||
UPDATE_ON_START=true
|
|
||||||
DELETE_ON_STOP=false
|
|
||||||
DELETE_ON_FAILURE=true
|
|
||||||
TTL=1
|
|
||||||
PROXIED=!is(dns.example.com) && !is(mc.example.com) && !is(ssh.example.com)
|
|
||||||
EMOJI=true
|
|
||||||
UPTIMEKUMA=https://uptime-kuma.example.com/api/push/AsaSDFGFkfklaFALSKffkfFKfkfkfkFK?status=up&msg=OK&ping=
|
|
||||||
REJECT_CLOUDFLARE_IPS=true
|
|
||||||
+7
-24
@@ -1,30 +1,13 @@
|
|||||||
services:
|
services:
|
||||||
cloudflare-ddns:
|
cloudflare-ddns:
|
||||||
image: timothyjmiller/cloudflare-ddns:2.2.0
|
image: timothyjmiller/cloudflare-ddns:latest
|
||||||
container_name: cloudflare-ddns
|
container_name: cloudflare-ddns
|
||||||
restart: unless-stopped
|
|
||||||
security_opt:
|
security_opt:
|
||||||
- no-new-privileges:true
|
- no-new-privileges:true
|
||||||
network_mode: "host"
|
network_mode: 'host'
|
||||||
# https://github.com/timothymiller/cloudflare-ddns#-quick-start
|
|
||||||
environment:
|
environment:
|
||||||
- CLOUDFLARE_API_TOKEN=${CLOUDFLARE_API_TOKEN:?Cloudflare API token is required}
|
- PUID=1000
|
||||||
- DOMAINS=${DOMAINS:-}
|
- PGID=1000
|
||||||
- IP4_DOMAINS=${IP4_DOMAINS:-}
|
volumes:
|
||||||
- IP6_DOMAINS=${IP6_DOMAINS:-}
|
- ./config.json:/config.json
|
||||||
- IP4_PROVIDER=${IP4_PROVIDER:-cloudflare.trace}
|
restart: unless-stopped
|
||||||
- IP6_PROVIDER=${IP6_PROVIDER:-none}
|
|
||||||
- UPDATE_CRON=${UPDATE_CRON:-@every 5m}
|
|
||||||
- UPDATE_ON_START=${UPDATE_ON_START:-true}
|
|
||||||
- DELETE_ON_STOP=${DELETE_ON_STOP:-false}
|
|
||||||
- DELETE_ON_FAILURE=${DELETE_ON_FAILURE:-true}
|
|
||||||
- TTL=${TTL:-1} # 1=auto
|
|
||||||
# to proxy only "dns.example.com" and "wfs.example.com" use "!is(dns.domain.com) && !is (wfs.domain.com)"
|
|
||||||
- PROXIED=${PROXIED:-true}
|
|
||||||
- EMOJI=${EMOJI:-true}
|
|
||||||
- UPTIMEKUMA=${UPTIMEKUMA:-}
|
|
||||||
- HEALTHCHECKS=${HEALTHCHECKS:-}
|
|
||||||
- REJECT_CLOUDFLARE_IPS=${REJECT_CLOUDFLARE_IPS:-true}
|
|
||||||
# volumes:
|
|
||||||
# Prefer using environment variables for configuration, config.json legacy support
|
|
||||||
# - ./config.json:/config.json
|
|
||||||
@@ -7,7 +7,7 @@
|
|||||||
"api_key": {
|
"api_key": {
|
||||||
"api_key": "api_key_here",
|
"api_key": "api_key_here",
|
||||||
"account_email": "your_email_here"
|
"account_email": "your_email_here"
|
||||||
},
|
}
|
||||||
"zone_id": "your_zone-id",
|
"zone_id": "your_zone-id",
|
||||||
"subdomains": [
|
"subdomains": [
|
||||||
{ "name": "", "proxied": true },
|
{ "name": "", "proxied": true },
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
secret.yaml
|
|
||||||
Whitespace-only changes.
@@ -1,36 +0,0 @@
|
|||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
reloader.stakater.com/auto: "true"
|
|
||||||
name: cfddns
|
|
||||||
labels:
|
|
||||||
app: cfddns
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: cfddns
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: cfddns
|
|
||||||
spec:
|
|
||||||
hostNetwork: true
|
|
||||||
dnsPolicy: ClusterFirstWithHostNet
|
|
||||||
containers:
|
|
||||||
- name: cloudflare-ddns
|
|
||||||
image: timothyjmiller/cloudflare-ddns:2.2.0
|
|
||||||
imagePullPolicy: Always
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "32Mi"
|
|
||||||
cpu: "30m"
|
|
||||||
limits:
|
|
||||||
memory: "128Mi"
|
|
||||||
cpu: "50m"
|
|
||||||
envFrom:
|
|
||||||
- secretRef:
|
|
||||||
name: cfddns-secrets
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: cfddns-secrets
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
CLOUDFLARE_API_TOKEN: your_token
|
|
||||||
DOMAINS: "example.com,www.example.com"
|
|
||||||
IP4_PROVIDER: cloudflare.trace
|
|
||||||
IP6_PROVIDER: none
|
|
||||||
UPDATE_CRON: "@every 5m"
|
|
||||||
UPDATE_ON_START: "true"
|
|
||||||
DELETE_ON_STOP: "false"
|
|
||||||
DELETE_ON_FAILURE: "true"
|
|
||||||
TTL: "1"
|
|
||||||
PROXIED: "true"
|
|
||||||
EMOJI: "true"
|
|
||||||
REJECT_CLOUDFLARE_IPS: "true"
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
CMK_PASSWORD=password
|
|
||||||
TZ=Europe/Berlin
|
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
services:
|
|
||||||
checkmk:
|
|
||||||
image: "checkmk/check-mk-raw:2.4.0-2026.09.14"
|
|
||||||
container_name: "checkmk"
|
|
||||||
restart: unless-stopped
|
|
||||||
# ports:
|
|
||||||
# - 5000:5000
|
|
||||||
# - 6776:8000
|
|
||||||
volumes:
|
|
||||||
- sites:/omd/sites
|
|
||||||
tmpfs:
|
|
||||||
- /opt/omd/sites/cmk/tmp:uid=1000,gid=1000
|
|
||||||
environment:
|
|
||||||
- CMK_PASSWORD=${CMK_PASSWORD:-password}
|
|
||||||
- CMK_SITE_ID=cmk
|
|
||||||
- TZ=${TZ:-Etc/UTC}
|
|
||||||
labels:
|
|
||||||
- "traefik.enable=true"
|
|
||||||
- "traefik.http.services.checkmk.loadbalancer.server.port=5000"
|
|
||||||
|
|
||||||
# Prod Router
|
|
||||||
- "traefik.http.routers.checkmk.rule=Host(`cmk.forust.xyz`)"
|
|
||||||
- "traefik.http.routers.checkmk.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.checkmk.tls.certresolver=letsencrypt"
|
|
||||||
# Local Router
|
|
||||||
- "traefik.http.routers.checkmk-local.rule=Host(`cmk.workstation.internal`)"
|
|
||||||
- "traefik.http.routers.checkmk-local.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.checkmk-local.tls=true"
|
|
||||||
# Dev Router
|
|
||||||
- "traefik.http.routers.checkmk-dev.rule=Host(`cmk.gigaforust.internal`)"
|
|
||||||
- "traefik.http.routers.checkmk-dev.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.checkmk-dev.tls=true"
|
|
||||||
networks:
|
|
||||||
- proxy
|
|
||||||
networks:
|
|
||||||
proxy:
|
|
||||||
external: true
|
|
||||||
volumes:
|
|
||||||
sites:
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: checkmk-prod-tls
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
secretName: checkmk-prod-tls
|
|
||||||
dnsNames:
|
|
||||||
- cmk.forust.xyz
|
|
||||||
issuerRef:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
kind: ClusterIssuer
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: internal-wildcard-tls
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
dnsNames:
|
|
||||||
- "*.workstation.internal"
|
|
||||||
- "*.gigaforust.internal"
|
|
||||||
- workstation.internal
|
|
||||||
- gigaforust.internal
|
|
||||||
issuerRef:
|
|
||||||
name: internal-ca
|
|
||||||
kind: ClusterIssuer
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: checkmk-service
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: checkmk
|
|
||||||
ports:
|
|
||||||
- name: web
|
|
||||||
port: 5000
|
|
||||||
targetPort: 5000
|
|
||||||
- name: agent-receiver
|
|
||||||
port: 8000
|
|
||||||
targetPort: 8000
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
reloader.stakater.com/auto: "true"
|
|
||||||
name: checkmk-deployment
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: checkmk
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: checkmk
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: checkmk
|
|
||||||
image: checkmk/check-mk-raw:2.4.0-2026.09.14
|
|
||||||
envFrom:
|
|
||||||
- secretRef:
|
|
||||||
name: checkmk-secrets
|
|
||||||
- configMapRef:
|
|
||||||
name: checkmk-config
|
|
||||||
ports:
|
|
||||||
- name: web
|
|
||||||
containerPort: 5000
|
|
||||||
- name: agent-receiver
|
|
||||||
containerPort: 8000
|
|
||||||
volumeMounts:
|
|
||||||
- name: sites
|
|
||||||
mountPath: /omd/sites
|
|
||||||
- name: tmp
|
|
||||||
mountPath: /opt/omd/sites/cmk/tmp
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "2Gi"
|
|
||||||
cpu: "600m"
|
|
||||||
limits:
|
|
||||||
memory: "5Gi"
|
|
||||||
cpu: "4"
|
|
||||||
volumes:
|
|
||||||
- name: sites
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: checkmk-sites-pvc
|
|
||||||
- name: tmp
|
|
||||||
emptyDir:
|
|
||||||
medium: Memory
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: checkmk-sites-pvc
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 5Gi
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: checkmk-config
|
|
||||||
namespace: checkmk
|
|
||||||
data:
|
|
||||||
TZ: Europe/Bratislava
|
|
||||||
CMK_SITE_ID: cmk
|
|
||||||
@@ -1,49 +0,0 @@
|
|||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: checkmk-prod
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`cmk.forust.xyz`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: checkmk-service
|
|
||||||
port: 5000
|
|
||||||
tls:
|
|
||||||
secretName: checkmk-prod-tls
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRouteTCP
|
|
||||||
metadata:
|
|
||||||
name: checkmk-agent-receiver
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- checkmk-agent
|
|
||||||
routes:
|
|
||||||
- match: HostSNI(`*`)
|
|
||||||
services:
|
|
||||||
- name: checkmk-service
|
|
||||||
port: 8000
|
|
||||||
tls:
|
|
||||||
passthrough: true
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: checkmk-local
|
|
||||||
namespace: checkmk
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`cmk.workstation.internal`) || Host(`cmk.gigaforust.internal`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: checkmk-service
|
|
||||||
port: 5000
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: checkmk
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: checkmk-secrets
|
|
||||||
namespace: checkmk
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
CMK_PASSWORD: "password"
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
secret.yaml
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
reloader.stakater.com/auto: "true"
|
|
||||||
name: cloudflared
|
|
||||||
labels:
|
|
||||||
app: cloudflared
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: cloudflared
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: cloudflared
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: cloudflared
|
|
||||||
image: cloudflare/cloudflared:2026.10.0
|
|
||||||
imagePullPolicy: IfNotPresent
|
|
||||||
args:
|
|
||||||
- tunnel
|
|
||||||
- --no-autoupdate
|
|
||||||
- run
|
|
||||||
env:
|
|
||||||
- name: TUNNEL_TOKEN
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: cloudflared-secrets
|
|
||||||
key: TUNNEL_TOKEN
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "128Mi"
|
|
||||||
cpu: "30m"
|
|
||||||
limits:
|
|
||||||
memory: "256Mi"
|
|
||||||
cpu: "200m"
|
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: cloudflared-secrets
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
TUNNEL_TOKEN: your_tunnel_token_here
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
ACCOUNT_REGISTRATION=false
|
|
||||||
HTTP_ALLOWED=false
|
|
||||||
ALLOW_UNAUTHENTICAED=false
|
|
||||||
AUTO_DELETE_EVERY_N_HOURS=24
|
|
||||||
WEBROOT=/convert
|
|
||||||
HIDE_HISTORY=false
|
|
||||||
LANGUAGE=en
|
|
||||||
UNAUTHED_USER_SHARING=false
|
|
||||||
MAX_CONVERT_PROCESS=0
|
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
services:
|
|
||||||
convertx:
|
|
||||||
container_name: convertx
|
|
||||||
image: ghcr.io/c4illin/convertx:v0.19.0
|
|
||||||
restart: unless-stopped
|
|
||||||
ports:
|
|
||||||
- "9992:3000"
|
|
||||||
# https://github.com/C4illin/ConvertX#environment-variables
|
|
||||||
environment:
|
|
||||||
- JWT_SECRET=$(JWT_SECRET)
|
|
||||||
- ACCOUNT_REGISTRATION=$(ACCOUNT_REGISTRATION:-false)
|
|
||||||
- HTTP_ALLOWED=$(HTTP_ALLOWED:-false)
|
|
||||||
- ALLOW_UNAUTHENTICATED=$(ALLOW_UNAUTHENTICATED:-false)
|
|
||||||
- AUTO_DELETE_EVERY_N_HOURS=$(AUTO_DELETE_EVERY_N_HOURS:-24)
|
|
||||||
- WEBROOT=$(WEBROOT)
|
|
||||||
- HIDE_HISTORY=$(HIDE_HISTORY:-false)
|
|
||||||
- LANGUAGE=$(LANGUAGE:-en)
|
|
||||||
- UNAUTHENTICATED_USER_SHARING=$(UNAUTHENTICATED_USER_SHARING:-false)
|
|
||||||
- MAX_CONVERT_PROCESS=$(MAX_CONVERT_PROCESS:-0)
|
|
||||||
labels:
|
|
||||||
- "traefik.enable=true"
|
|
||||||
- "traefik.http.services.convertx.loadbalancer.server.port=3000"
|
|
||||||
# Prod Router
|
|
||||||
- "traefik.http.routers.convertx.rule=(Host(`forust.xyz`) || Host(`www.forust.xyz`)) && PathPrefix(`/convert`)"
|
|
||||||
- "traefik.http.routers.convertx.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.convertx.priority=50"
|
|
||||||
- "traefik.http.routers.convertx.tls.certresolver=letsencrypt"
|
|
||||||
# Local Router
|
|
||||||
- "traefik.http.routers.convertx-local.rule=Host(`workstation.internal`) && PathPrefix(`/convert`)"
|
|
||||||
- "traefik.http.routers.convertx-local.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.convertx-local.priority=50"
|
|
||||||
- "traefik.http.routers.convertx-local.tls=true"
|
|
||||||
# Dev Router
|
|
||||||
- "traefik.http.routers.convertx-dev.rule=Host(`gigaforust.internal`) && PathPrefix(`/convert`)"
|
|
||||||
- "traefik.http.routers.convertx-dev.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.convertx-dev.priority=50"
|
|
||||||
- "traefik.http.routers.convertx-dev.tls=true"
|
|
||||||
networks:
|
|
||||||
- proxy
|
|
||||||
volumes:
|
|
||||||
- data:/app/data
|
|
||||||
|
|
||||||
bentopdf:
|
|
||||||
container_name: bentopdf
|
|
||||||
image: bentopdf/bentopdf@sha256:4eb4ec8f5030faf87c29a73d3d5a2781f28a597cf440c3ab111eb96aee550871
|
|
||||||
restart: unless-stopped
|
|
||||||
labels:
|
|
||||||
- "traefik.enable=true"
|
|
||||||
- "traefik.http.services.bentopdf.loadbalancer.server.port=8080"
|
|
||||||
|
|
||||||
# Prod router
|
|
||||||
- "traefik.http.routers.bentopdf.rule=Host(`pdf.forust.xyz`)"
|
|
||||||
- "traefik.http.routers.bentopdf.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.bentopdf.tls.certresolver=letsencrypt"
|
|
||||||
- "traefik.http.routers.bentopdf.tls=true"
|
|
||||||
# Local router
|
|
||||||
- "traefik.http.routers.bentopdf-local.rule=Host(`pdf.workstation.internal`)"
|
|
||||||
- "traefik.http.routers.bentopdf-local.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.bentopdf-local.tls=true"
|
|
||||||
# Dev router
|
|
||||||
- "traefik.http.routers.bentopdf-dev.rule=Host(`pdf.gigaforust.internal`)"
|
|
||||||
- "traefik.http.routers.bentopdf-dev.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.bentopdf-dev.tls=true"
|
|
||||||
networks:
|
|
||||||
- proxy
|
|
||||||
networks:
|
|
||||||
proxy:
|
|
||||||
external: true
|
|
||||||
volumes:
|
|
||||||
data:
|
|
||||||
Whitespace-only changes.
@@ -1,43 +0,0 @@
|
|||||||
kind: Service
|
|
||||||
apiVersion: v1
|
|
||||||
metadata:
|
|
||||||
name: bentopdf-service
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: bentopdf
|
|
||||||
ports:
|
|
||||||
- port: 8080
|
|
||||||
targetPort: 8080
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: bentopdf-deployment
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
replicas: 2
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: bentopdf
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: bentopdf
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- image: bentopdf/bentopdf@sha256:4eb4ec8f5030faf87c29a73d3d5a2781f28a597cf440c3ab111eb96aee550871
|
|
||||||
imagePullPolicy: Always
|
|
||||||
name: bentopdf
|
|
||||||
ports:
|
|
||||||
- containerPort: 8080
|
|
||||||
# p95 4M, max 11M over 7 days. Was 50Mi/700Mi.
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "32Mi"
|
|
||||||
cpu: "50m"
|
|
||||||
ephemeral-storage: "100Mi"
|
|
||||||
limits:
|
|
||||||
memory: "128Mi"
|
|
||||||
cpu: "700m"
|
|
||||||
ephemeral-storage: "5Gi"
|
|
||||||
@@ -1,42 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: convertx-prod-tls
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
secretName: convertx-prod-tls
|
|
||||||
dnsNames:
|
|
||||||
- forust.xyz
|
|
||||||
- www.forust.xyz
|
|
||||||
issuerRef:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
kind: ClusterIssuer
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: bentopdf-prod-tls
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
secretName: bentopdf-prod-tls
|
|
||||||
dnsNames:
|
|
||||||
- pdf.forust.xyz
|
|
||||||
issuerRef:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
kind: ClusterIssuer
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: internal-wildcard-tls
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
dnsNames:
|
|
||||||
- "*.workstation.internal"
|
|
||||||
- "*.gigaforust.internal"
|
|
||||||
- workstation.internal
|
|
||||||
- gigaforust.internal
|
|
||||||
issuerRef:
|
|
||||||
name: internal-ca
|
|
||||||
kind: ClusterIssuer
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
# test manifest with docker and k8s config keys mismatch
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: convertx-config
|
|
||||||
namespace: converters
|
|
||||||
data:
|
|
||||||
ACCOUNT_REGISTRATION: "false"
|
|
||||||
HTTP_ALLOWED: "false"
|
|
||||||
ALLOW_UNAUTHENTICAED: "false"
|
|
||||||
AUTO_DELETE_EVERY_N_HOURS: "24"
|
|
||||||
WEBROOT: "/convert"
|
|
||||||
HIDE_HISTORY: "false"
|
|
||||||
LANGUAGE: "en"
|
|
||||||
UNAUTHED_USER_SHARING: "false"
|
|
||||||
MAX_CONVERT_PROCESS: "0"
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: convertx-service
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: convertx
|
|
||||||
ports:
|
|
||||||
- port: 3000
|
|
||||||
targetPort: 3000
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
annotations:
|
|
||||||
reloader.stakater.com/auto: "true"
|
|
||||||
name: convertx-deployment
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: convertx
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: convertx
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- image: ghcr.io/c4illin/convertx:v0.19.0
|
|
||||||
name: convertx
|
|
||||||
envFrom:
|
|
||||||
- configMapRef:
|
|
||||||
name: convertx-config
|
|
||||||
- secretRef:
|
|
||||||
name: convertx-secrets
|
|
||||||
ports:
|
|
||||||
- containerPort: 3000
|
|
||||||
volumeMounts:
|
|
||||||
- mountPath: /data
|
|
||||||
name: data
|
|
||||||
# p95 85M, max 136M over 7 days, spikes while converting. Was 250Mi/1.5Gi.
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "128Mi"
|
|
||||||
cpu: "100m"
|
|
||||||
limits:
|
|
||||||
cpu: "1500m"
|
|
||||||
memory: "512Mi"
|
|
||||||
volumes:
|
|
||||||
- name: data
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: convertx-pvc
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: convertx-pvc
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 2Gi
|
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: convertx-prod
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: (Host(`forust.xyz`) || Host(`www.forust.xyz`)) && PathPrefix(`/convert`)
|
|
||||||
kind: Rule
|
|
||||||
priority: 50
|
|
||||||
services:
|
|
||||||
- name: convertx-service
|
|
||||||
port: 3000
|
|
||||||
tls:
|
|
||||||
secretName: convertx-prod-tls
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: convertx-local
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: (Host(`workstation.internal`) || Host(`gigaforust.internal`)) && PathPrefix(`/convert`)
|
|
||||||
kind: Rule
|
|
||||||
priority: 50
|
|
||||||
services:
|
|
||||||
- name: convertx-service
|
|
||||||
port: 3000
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: bentopdf-prod
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`pdf.forust.xyz`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: bentopdf-service
|
|
||||||
port: 8080
|
|
||||||
tls:
|
|
||||||
secretName: bentopdf-prod-tls
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: bentopdf-local
|
|
||||||
namespace: converters
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`pdf.workstation.internal`) || Host(`pdf.gigaforust.internal`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: bentopdf-service
|
|
||||||
port: 8080
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: converters
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: convertx-secrets
|
|
||||||
namespace: converters
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
jwt-secret: ""
|
|
||||||
@@ -1,203 +0,0 @@
|
|||||||
container_runtime: containerd
|
|
||||||
|
|
||||||
agent:
|
|
||||||
acquisition: []
|
|
||||||
additionalAcquisition:
|
|
||||||
- labels:
|
|
||||||
type: traefik
|
|
||||||
limit: 1000
|
|
||||||
query: |
|
|
||||||
{namespace="traefik"}
|
|
||||||
source: loki
|
|
||||||
url: http://loki.prometheus.svc.cluster.local:3100/
|
|
||||||
wait_for_ready: 30s
|
|
||||||
env:
|
|
||||||
- name: COLLECTIONS
|
|
||||||
value: crowdsecurity/traefik crowdsecurity/base-http-scenarios
|
|
||||||
- name: DISABLE_COLLECTIONS
|
|
||||||
value: crowdsecurity/sshd
|
|
||||||
# Bans on 401/403 bursts hurt more than they protect: with L3 enforcement
|
|
||||||
# a false positive cuts the IP off everything (SSH included), and past
|
|
||||||
# incidents show legit automation (deploy runner, mesh peers, registry
|
|
||||||
# pulls) tripping this probe. Probing/XSS/SQLi/CVE scenarios stay.
|
|
||||||
- name: DISABLE_SCENARIOS
|
|
||||||
value: crowdsecurity/http-generic-bf
|
|
||||||
metrics:
|
|
||||||
enabled: true
|
|
||||||
serviceMonitor:
|
|
||||||
additionalLabels:
|
|
||||||
release: prometheus-stack
|
|
||||||
enabled: true
|
|
||||||
# Static machine identity: agent pods mount pre-created LAPI credentials
|
|
||||||
# (Secret crowdsec-agent-credentials, key local_api_credentials.yaml)
|
|
||||||
# at the exact path the agent entrypoint expects. Together with the
|
|
||||||
# patched register-init (enforced by janitor-cronjob.yaml) the agent
|
|
||||||
# never calls `cscli lapi register` in steady state, so pod names,
|
|
||||||
# restarts and reboots can no longer break it.
|
|
||||||
extraVolumes:
|
|
||||||
- name: static-creds
|
|
||||||
secret:
|
|
||||||
secretName: crowdsec-agent-credentials
|
|
||||||
items:
|
|
||||||
- key: local_api_credentials.yaml
|
|
||||||
path: local_api_credentials.yaml
|
|
||||||
extraVolumeMounts:
|
|
||||||
- name: static-creds
|
|
||||||
mountPath: /tmp_config/local_api_credentials.yaml
|
|
||||||
subPath: local_api_credentials.yaml
|
|
||||||
readOnly: true
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpu: 200m
|
|
||||||
memory: 500Mi
|
|
||||||
requests:
|
|
||||||
cpu: 50m
|
|
||||||
memory: 100Mi
|
|
||||||
|
|
||||||
config:
|
|
||||||
parsers:
|
|
||||||
s02-enrich:
|
|
||||||
mobile-whitelist.yaml: |
|
|
||||||
name: forust/mobile-whitelist
|
|
||||||
description: "Whitelist SWAN/4ka mobile network"
|
|
||||||
whitelist:
|
|
||||||
reason: "Mobile IP whitelist"
|
|
||||||
cidr:
|
|
||||||
- "84.245.64.0/18"
|
|
||||||
# CrowdSec's own guidance: CIDR allowlisting belongs at the parser stage.
|
|
||||||
# A parser whitelist discards the event before it reaches a bucket, so
|
|
||||||
# these addresses never produce an overflow and never become a decision.
|
|
||||||
# A postoverflow whitelist is checked only *after* the ban exists, and
|
|
||||||
# the bouncer answers 403 for as long as it does - which is a window we
|
|
||||||
# do not want the deploy sitting in.
|
|
||||||
local-network.yaml: |
|
|
||||||
name: forust/local-network
|
|
||||||
description: "Whitelist loopback, private and VPN networks"
|
|
||||||
whitelist:
|
|
||||||
reason: "Local network"
|
|
||||||
cidr:
|
|
||||||
- "127.0.0.0/8"
|
|
||||||
- "10.0.0.0/8"
|
|
||||||
- "172.16.0.0/12"
|
|
||||||
- "192.168.0.0/16"
|
|
||||||
# CGNAT range (RFC 6598). The workstation and the k0s node live
|
|
||||||
# here on WireGuard, and 100.64.0.0/10 is not covered by the
|
|
||||||
# RFC 1918 blocks above.
|
|
||||||
- "100.64.0.0/10"
|
|
||||||
- "169.254.0.0/16"
|
|
||||||
- "fc00::/7"
|
|
||||||
- "fe80::/10"
|
|
||||||
vps-whitelist.yaml: |
|
|
||||||
name: forust/vps-whitelist
|
|
||||||
description: "Whitelist static VPS"
|
|
||||||
whitelist:
|
|
||||||
reason: "VPS"
|
|
||||||
ip:
|
|
||||||
- "193.181.211.79"
|
|
||||||
|
|
||||||
postoverflows:
|
|
||||||
s01-whitelist:
|
|
||||||
# The one whitelist that has to stay here: resolving a hostname is a
|
|
||||||
# network call, and the docs put expensive lookups in postoverflows on
|
|
||||||
# purpose - it runs only when a bucket actually overflows.
|
|
||||||
# ddns.forust.xyz is the public home address, not a private one, so
|
|
||||||
# forust/local-network does not cover it.
|
|
||||||
home-dynamic-ip.yaml: |
|
|
||||||
name: forust/home-dynamic-ip
|
|
||||||
description: "Whitelist home dynamic IP"
|
|
||||||
whitelist:
|
|
||||||
reason: "Home dynamic IP"
|
|
||||||
expression:
|
|
||||||
- evt.Overflow.Alert.Source.IP in LookupHost("ddns.forust.xyz")
|
|
||||||
|
|
||||||
# LAPI-only main config override, merged over config.yaml. NOTE: the
|
|
||||||
# chart's own default for this key is REPLACED, not merged, so its
|
|
||||||
# auto_registration block is repeated verbatim below - drop it and the
|
|
||||||
# agent can no longer register itself.
|
|
||||||
config.yaml.local: |
|
|
||||||
api:
|
|
||||||
server:
|
|
||||||
auto_registration: # Activate if not using TLS for authentication
|
|
||||||
enabled: true
|
|
||||||
token: "${REGISTRATION_TOKEN}" # /!\ Do not modify this variable (auto-generated and handled by the chart)
|
|
||||||
allowed_ranges: # /!\ Make sure to adapt to the pod IP ranges used by your cluster
|
|
||||||
- "127.0.0.1/32"
|
|
||||||
- "192.168.0.0/16"
|
|
||||||
- "10.0.0.0/8"
|
|
||||||
- "172.16.0.0/12"
|
|
||||||
# This homelab has no egress to console.crowdsec.cloud: DNS does
|
|
||||||
# not resolve. The LAPI kept trying anyway ("Signal push: N
|
|
||||||
# signals to push", "capi metrics: sending" every 10s) and each
|
|
||||||
# attempt sat on a resolver timeout WHILE HOLDING A WRITE
|
|
||||||
# TRANSACTION, which is what kept stalling per-request decision
|
|
||||||
# lookups even with WAL enabled. Nothing to share and nothing to
|
|
||||||
# pull - turn the Central API off instead of letting it block the
|
|
||||||
# only database writer we have.
|
|
||||||
online_client:
|
|
||||||
sharing: false
|
|
||||||
pull:
|
|
||||||
community: false
|
|
||||||
blocklists: false
|
|
||||||
disable_usage_metrics_export: true
|
|
||||||
db_config:
|
|
||||||
# SQLite without WAL serialises every reader behind the writer's
|
|
||||||
# rollback journal, and the LAPI writes constantly: the agent pushes
|
|
||||||
# Traefik alerts read from Loki, the metrics collector counts
|
|
||||||
# decisions, the bouncer touches "last pull" on every request.
|
|
||||||
# Symptom: decision lookups taking 10-30s (and a second connection
|
|
||||||
# that could not even open the database) while the LAPI sat at 28m
|
|
||||||
# CPU - the process was blocked in fsync, not computing. Every
|
|
||||||
# bouncer-protected request then blew through the plugin timeout and
|
|
||||||
# fail-closed with 403, on every site at once.
|
|
||||||
# The PVC is local-path-retain (hostPath), not a network share, so
|
|
||||||
# WAL is safe here; the crowdsec docs recommend it for exactly this
|
|
||||||
# ("allowing more concurrency in SQLite that will improve
|
|
||||||
# performances in most scenarios").
|
|
||||||
use_wal: true
|
|
||||||
# Keeps the alert table bounded. At the 5000/7d default the file
|
|
||||||
# reached 54MB in 15 days off the Traefik access log alone, and the
|
|
||||||
# metrics collector counts decisions on a timer; a smaller working
|
|
||||||
# set means fewer full scans. Crowdsec only prunes - SQLite never
|
|
||||||
# shrinks the file, so the size stays until a manual VACUUM.
|
|
||||||
flush:
|
|
||||||
max_items: 1000
|
|
||||||
max_age: 24h
|
|
||||||
|
|
||||||
lapi:
|
|
||||||
env:
|
|
||||||
- name: COLLECTIONS
|
|
||||||
value: crowdsecurity/traefik crowdsecurity/base-http-scenarios
|
|
||||||
- name: DISABLE_COLLECTIONS
|
|
||||||
value: crowdsecurity/linux crowdsecurity/sshd
|
|
||||||
metrics:
|
|
||||||
enabled: true
|
|
||||||
serviceMonitor:
|
|
||||||
additionalLabels:
|
|
||||||
release: prometheus-stack
|
|
||||||
enabled: true
|
|
||||||
persistentVolume:
|
|
||||||
config:
|
|
||||||
enabled: true
|
|
||||||
size: 100Mi
|
|
||||||
storageClassName: local-path-retain
|
|
||||||
data:
|
|
||||||
enabled: true
|
|
||||||
size: 1Gi
|
|
||||||
storageClassName: local-path-retain
|
|
||||||
# LAPI answers a blocking /v1/decisions lookup for EVERY bouncer-protected
|
|
||||||
# request (whole Traefik front door), so it is the hot path of the proxy.
|
|
||||||
# At 400m/500Mi it went CPU-throttled and idle lookups measured 1.3-7.4s,
|
|
||||||
# which pushed requests into the bouncer's fail-closed 403.
|
|
||||||
# Single replica on purpose: LAPI is stateful (BoltDB on the `data` PVC,
|
|
||||||
# credentials on the `config` PVC) - two replicas sharing those RWO
|
|
||||||
# volumes would corrupt the decision store. Scale up CPU, not replicas.
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpu: 1500m
|
|
||||||
memory: 1Gi
|
|
||||||
requests:
|
|
||||||
cpu: 250m
|
|
||||||
memory: 500Mi
|
|
||||||
service:
|
|
||||||
type: ClusterIP
|
|
||||||
storeLAPICscliCredentialsInSecret: true
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
data:
|
|
||||||
crowdsec-overview.json: "{\n \"__inputs\": [\n {\n \"name\": \"DS_PROMETHEUS\",\n \"label\": \"Prometheus\",\n \"description\": \"\",\n \"type\": \"datasource\",\n \"pluginId\": \"prometheus\",\n \"pluginName\": \"Prometheus\"\n }\n ],\n \"__requires\": [\n {\n \"type\": \"grafana\",\n \"id\": \"grafana\",\n \"name\": \"Grafana\",\n \"version\": \"8.1.2\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"graph\",\n \"name\": \"Graph (old)\",\n \"version\": \"\"\n },\n {\n \"type\": \"datasource\",\n \"id\": \"prometheus\",\n \"name\": \"Prometheus\",\n \"version\": \"1.0.0\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"stat\",\n \"name\": \"Stat\",\n \"version\": \"\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"timeseries\",\n \"name\": \"Time series\",\n \"version\": \"\"\n }\n ],\n \"annotations\": {\n \"list\": [\n {\n \"builtIn\": 1,\n \"datasource\": \"-- Grafana --\",\n \"enable\": true,\n \"hide\": true,\n \"iconColor\": \"rgba(0, 211, 255, 1)\",\n \"name\": \"Annotations & Alerts\",\n \"target\": {\n \"limit\": 100,\n \"matchAny\": false,\n \"tags\": [],\n \"type\": \"dashboard\"\n },\n \"type\": \"dashboard\"\n }\n ]\n },\n \"editable\": true,\n \"gnetId\": null,\n \"graphTooltip\": 0,\n \"id\": null,\n \"links\": [],\n \"panels\": [\n {\n \"collapsed\": false,\n \"datasource\": null,\n \"gridPos\": {\n \"h\": 1,\n \"w\": 24,\n \"x\": 0,\n \"y\": 0\n },\n \"id\": 24,\n \"panels\": [],\n \"title\": \"Summary\",\n \"type\": \"row\"\n },\n {\n \"cacheTimeout\": null,\n \"datasource\": \"${DS_PROMETHEUS}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [\n {\n \"options\": {\n \"match\": \"null\",\n \"result\": {\n \"text\": \"N/A\"\n }\n },\n \"type\": \"special\"\n }\n ],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"#E02F44\",\n \"value\": null\n },\n {\n \"color\": \"#E02F44\",\n \"value\": 10\n },\n {\n \"color\": \"#299c46\",\n \"value\": 10\n }\n ]\n },\n \"unit\": \"none\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 6,\n \"x\": 0,\n \"y\": 1\n },\n \"id\": 2,\n \"interval\": null,\n \"links\": [],\n \"maxDataPoints\": 100,\n \"options\": {\n \"colorMode\": \"background\",\n \"graphMode\": \"none\",\n \"justifyMode\": \"auto\",\n \"orientation\": \"horizontal\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"text\": {},\n \"textMode\": \"auto\"\n },\n \"pluginVersion\": \"8.1.2\",\n \"targets\": [\n {\n \"exemplar\": true,\n \"expr\": \"count(cs_info)\",\n \"interval\": \"\",\n \"legendFormat\": \"\",\n \"refId\": \"A\"\n }\n ],\n \"timeFrom\": null,\n \"timeShift\": null,\n \"title\": \"Running Crowdsec\",\n \"transparent\": true,\n \"type\": \"stat\"\n },\n {\n \"aliasColors\": {},\n \"bars\": false,\n \"dashLength\": 10,\n \"dashes\": false,\n \"datasource\": \"${DS_PROMETHEUS}\",\n \"decimals\": 1,\n \"fieldConfig\": {\n \"defaults\": {\n \"links\": []\n },\n \"overrides\": []\n },\n \"fill\": 1,\n \"fillGradient\": 0,\n \"gridPos\": {\n \"h\": 8,\n \"w\": 18,\n \"x\": 6,\n \"y\": 1\n },\n \"hiddenSeries\": false,\n \"id\": 8,\n \"legend\": {\n \"alignAsTable\": true,\n \"avg\": false,\n \"current\": false,\n \"max\": false,\n \"min\": false,\n \"rightSide\": true,\n \"show\": true,\n \"sort\": \"total\",\n \"sortDesc\": true,\n \"total\": true,\n \"values\": true\n },\n \"lines\": true,\n \"linewidth\": 1,\n \"nullPointMode\": \"null\",\n \"options\": {\n \"alertThreshold\": true\n },\n \"percentage\": false,\n \"pluginVersion\": \"8.1.2\",\n \"pointradius\": 2,\n \"points\": false,\n \"renLine truncated
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/managed-by: manual
|
|
||||||
grafana_dashboard: "1"
|
|
||||||
name: crowdsec-crowdsec-overview
|
|
||||||
namespace: prometheus
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
data:
|
|
||||||
crowdsec-lapi-metrics.json: "{\n \"__inputs\": [\n {\n \"name\": \"DS_PROMETHEUS\",\n \"label\": \"Prometheus\",\n \"description\": \"\",\n \"type\": \"datasource\",\n \"pluginId\": \"prometheus\",\n \"pluginName\": \"Prometheus\"\n }\n ],\n \"__requires\": [\n {\n \"type\": \"panel\",\n \"id\": \"bargauge\",\n \"name\": \"Bar gauge\",\n \"version\": \"\"\n },\n {\n \"type\": \"grafana\",\n \"id\": \"grafana\",\n \"name\": \"Grafana\",\n \"version\": \"8.1.2\"\n },\n {\n \"type\": \"datasource\",\n \"id\": \"prometheus\",\n \"name\": \"Prometheus\",\n \"version\": \"1.0.0\"\n }\n ],\n \"annotations\": {\n \"list\": [\n {\n \"builtIn\": 1,\n \"datasource\": \"-- Grafana --\",\n \"enable\": true,\n \"hide\": true,\n \"iconColor\": \"rgba(0, 211, 255, 1)\",\n \"name\": \"Annotations & Alerts\",\n \"target\": {\n \"limit\": 100,\n \"matchAny\": false,\n \"tags\": [],\n \"type\": \"dashboard\"\n },\n \"type\": \"dashboard\"\n }\n ]\n },\n \"editable\": true,\n \"gnetId\": null,\n \"graphTooltip\": 0,\n \"id\": null,\n \"iteration\": 1655915193937,\n \"links\": [],\n \"panels\": [\n {\n \"collapsed\": false,\n \"datasource\": null,\n \"gridPos\": {\n \"h\": 1,\n \"w\": 24,\n \"x\": 0,\n \"y\": 0\n },\n \"id\": 10,\n \"panels\": [],\n \"title\": \"Agents\",\n \"type\": \"row\"\n },\n {\n \"datasource\": \"${DS_PROMETHEUS}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n }\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 1\n },\n \"id\": 2,\n \"options\": {\n \"displayMode\": \"gradient\",\n \"orientation\": \"vertical\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"showUnfilled\": false,\n \"text\": {}\n },\n \"pluginVersion\": \"8.1.2\",\n \"repeat\": \"query0\",\n \"repeatDirection\": \"h\",\n \"targets\": [\n {\n \"exemplar\": false,\n \"expr\": \"sum(rate(cs_lapi_request_duration_seconds_bucket{endpoint=\\\"/v1/watchers/login\\\", instance=\\\"$lapi\\\"}[$__rate_interval])) by (le)\",\n \"format\": \"heatmap\",\n \"interval\": \"\",\n \"legendFormat\": \"{{le}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Agents Login\",\n \"type\": \"heatmap\"\n },\n {\n \"datasource\": \"${DS_PROMETHEUS}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n }\n ]\n },\n \"unit\": \"none\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 1\n },\n \"id\": 6,\n \"options\": {\n \"displayMode\": \"gradient\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"showUnfilled\": false,\n \"text\": {}\n },\n \"pluginVersion\": \"8.1.2\",\n \"targets\": [\n {\n \"exemplar\": true,\n \"expr\": \"sum(rate(cs_lapi_request_duration_seconds_bucket{endpoint=\\\"/v1/watchers/login\\\"}[$__rate_interval])) by (le)\",\n \"format\": \"heatmap\",\n \"interval\": \"\",\n \"legendFormat\": \"{{le}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Heartbeat\",\n \"type\": \"heatmap\"\n },\n {\n \"collapsed\": false,\n \"datasource\": null,\n \"gridPos\": {\n \"h\": 1,\n \"w\": 24,\n \"x\": 0,\n \"y\": 9\n },\n \"id\": 12,\n \"panels\": [],\n \"title\": \"Decisions\",\n \"type\": \"row\"\n },\n {\n \"datasource\": \"${DS_PROMETHEUS}\",\n Line truncated
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/managed-by: manual
|
|
||||||
grafana_dashboard: "1"
|
|
||||||
name: crowdsec-crowdsec-lapi-metrics
|
|
||||||
namespace: prometheus
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
data:
|
|
||||||
crowdsec-insight.json: "{\n \"__inputs\": [\n {\n \"name\": \"DS_PROMETHEUS\",\n \"label\": \"Prometheus\",\n \"description\": \"\",\n \"type\": \"datasource\",\n \"pluginId\": \"prometheus\",\n \"pluginName\": \"Prometheus\"\n }\n ],\n \"__requires\": [\n {\n \"type\": \"panel\",\n \"id\": \"bargauge\",\n \"name\": \"Bar gauge\",\n \"version\": \"\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"gauge\",\n \"name\": \"Gauge\",\n \"version\": \"\"\n },\n {\n \"type\": \"grafana\",\n \"id\": \"grafana\",\n \"name\": \"Grafana\",\n \"version\": \"8.1.2\"\n },\n {\n \"type\": \"datasource\",\n \"id\": \"prometheus\",\n \"name\": \"Prometheus\",\n \"version\": \"1.0.0\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"stat\",\n \"name\": \"Stat\",\n \"version\": \"\"\n }\n ],\n \"annotations\": {\n \"list\": [\n {\n \"builtIn\": 1,\n \"datasource\": \"-- Grafana --\",\n \"enable\": true,\n \"hide\": true,\n \"iconColor\": \"rgba(0, 211, 255, 1)\",\n \"name\": \"Annotations & Alerts\",\n \"target\": {\n \"limit\": 100,\n \"matchAny\": false,\n \"tags\": [],\n \"type\": \"dashboard\"\n },\n \"type\": \"dashboard\"\n }\n ]\n },\n \"editable\": true,\n \"gnetId\": null,\n \"graphTooltip\": 0,\n \"id\": null,\n \"iteration\": 1655915159751,\n \"links\": [],\n \"panels\": [\n {\n \"collapsed\": true,\n \"datasource\": null,\n \"gridPos\": {\n \"h\": 1,\n \"w\": 24,\n \"x\": 0,\n \"y\": 0\n },\n \"id\": 22,\n \"panels\": [\n {\n \"cacheTimeout\": null,\n \"datasource\": \"${DS_PROMETHEUS}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [\n {\n \"options\": {\n \"match\": \"null\",\n \"result\": {\n \"text\": \"N/A\"\n }\n },\n \"type\": \"special\"\n }\n ],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"dateTimeAsIso\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 9,\n \"w\": 5,\n \"x\": 2,\n \"y\": 1\n },\n \"id\": 2,\n \"interval\": null,\n \"links\": [],\n \"maxDataPoints\": 100,\n \"options\": {\n \"colorMode\": \"none\",\n \"graphMode\": \"none\",\n \"justifyMode\": \"auto\",\n \"orientation\": \"horizontal\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"text\": {},\n \"textMode\": \"auto\"\n },\n \"pluginVersion\": \"8.1.2\",\n \"targets\": [\n {\n \"exemplar\": true,\n \"expr\": \"(process_start_time_seconds{instance=\\\"$instance\\\"})*1000\",\n \"interval\": \"\",\n \"legendFormat\": \"{{instance}}\",\n \"refId\": \"A\"\n }\n ],\n \"timeFrom\": null,\n \"timeShift\": null,\n \"title\": \"Up since\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": \"${DS_PROMETHEUS}\",\n \"fieldConfig\": {\n \"defaults\": {\n \"displayName\": \"\",\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n }\n ]\n },\n \"unit\": \"decbytes\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 9,\n \"w\": 5,\n \"x\": 7,\n \"y\": 1\n },\n \"id\": 4,\n \"options\": {\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"calcs\": [\n \"mean\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\Line truncated
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/managed-by: manual
|
|
||||||
grafana_dashboard: "1"
|
|
||||||
name: crowdsec-crowdsec-insight
|
|
||||||
namespace: prometheus
|
|
||||||
@@ -1,202 +0,0 @@
|
|||||||
# CrowdSec self-healing: static machine identity + enforcement loops.
|
|
||||||
#
|
|
||||||
# Problem it fixes: the chart's agent init container runs
|
|
||||||
# `cscli lapi register --machine "$POD_NAME" ...`
|
|
||||||
# unconditionally. Credentials live in an emptyDir, the machine row lives
|
|
||||||
# in LAPI's persistent DB. Any init re-run for an already-known pod name
|
|
||||||
# (kubelet restart, node reboot) dies with
|
|
||||||
# 403 Forbidden: user '<pod>' already exist
|
|
||||||
# and the DaemonSet pod sticks in Init forever. Every DS restart also
|
|
||||||
# leaves an orphan machine row that is never cleaned.
|
|
||||||
#
|
|
||||||
# Design (name-independent):
|
|
||||||
# * Agent identity is a STATIC machine `crowdsec-agent-workstation`
|
|
||||||
# whose password lives in Secret `crowdsec-agent-credentials`
|
|
||||||
# (created once, manually - like all other secrets in this repo).
|
|
||||||
# The secret is mounted into agent pods at
|
|
||||||
# /tmp_config/local_api_credentials.yaml (see extraVolumeMounts in
|
|
||||||
# crowdsec-values.yaml), which is exactly the path the agent's main
|
|
||||||
# container copies into place at startup.
|
|
||||||
# * The DS init command is patched (strategic merge, by container name)
|
|
||||||
# to SKIP registration when that file exists, keeping the legacy
|
|
||||||
# register path only as fallback. Detection marker in the patched
|
|
||||||
# command: `[ -s /tmp_config`.
|
|
||||||
# * This CronJob enforces the desired state hourly, so recovery is
|
|
||||||
# automatic even after `helm upgrade` reverts the DS patch or the
|
|
||||||
# LAPI database is wiped:
|
|
||||||
# 1. patch DS init if it still has the unconditional register
|
|
||||||
# (no-op otherwise - no restart churn);
|
|
||||||
# 2. prune machines with no heartbeat for 2h (orphan hygiene);
|
|
||||||
# 3. ensure the static machine exists, recreating it with the
|
|
||||||
# Secret password if missing (agent retry loops reconnect
|
|
||||||
# on their own - same name + same password);
|
|
||||||
# 4. prune bouncer entries idle for 30d.
|
|
||||||
#
|
|
||||||
# It used to also delete LePresidente/http-generic-403-bf decisions hourly.
|
|
||||||
# That was a workaround for the bouncer failing closed on a slow LAPI and
|
|
||||||
# 403-ing the deploy runner into a 4h ban. The bouncer now polls decisions
|
|
||||||
# into a cache and never blocks on an unreachable LAPI, so it cannot
|
|
||||||
# manufacture those 403s any more, and the scenario only fires against real
|
|
||||||
# scanners - deleting their decisions hourly was undoing a working ban.
|
|
||||||
#
|
|
||||||
# Manual apply (crowdsec/k8s is NOT managed by deploy.yaml):
|
|
||||||
# kubectl apply -f crowdsec/k8s/janitor-cronjob.yaml
|
|
||||||
# Force a run:
|
|
||||||
# kubectl create job -n crowdsec --from=cronjob/crowdsec-janitor janitor-now
|
|
||||||
#
|
|
||||||
# Helm upgrades: the janitor's strategic patch puts the DS field under
|
|
||||||
# the `kubectl-patch` field manager, so a plain `helm upgrade` FAILS
|
|
||||||
# with an SSA conflict on initContainers[].command. Procedure:
|
|
||||||
# 1. revert init to chart state (kills the conflict):
|
|
||||||
# helm template crowdsec crowdsec/crowdsec --version <ver> \
|
|
||||||
# -n crowdsec -f crowdsec/k8s/crowdsec-values.yaml > /tmp/r.yaml
|
|
||||||
# python3 -c "import yaml,json; ..." # build revert patch from
|
|
||||||
# the rendered DaemonSet init command, then
|
|
||||||
# kubectl patch ds crowdsec-agent -n crowdsec \
|
|
||||||
# --type strategic -p "\$(cat /tmp/revert_patch.json)"
|
|
||||||
# 2. helm upgrade --install crowdsec ... (no --force needed)
|
|
||||||
# 3. janitor-now right away (upgrade reverts init; new pods would
|
|
||||||
# sit in Init until the next hourly run otherwise).
|
|
||||||
#
|
|
||||||
# One-time bootstrap (order matters):
|
|
||||||
# 1. Create Secret + static machine (see commands in chat).
|
|
||||||
# 2. Apply this file, trigger janitor-now, wait for agent 1/1.
|
|
||||||
# 3. One-time orphan cleanup:
|
|
||||||
# kubectl exec -n crowdsec deploy/crowdsec-lapi -- \
|
|
||||||
# cscli machines prune --duration 1h --force
|
|
||||||
# 4. Only then `helm upgrade` crowdsec with the extraVolumes values.
|
|
||||||
# Upgrade reverts the DS patch; trigger janitor-now right after it
|
|
||||||
# (otherwise new pods sit in Init until the next hourly run, then
|
|
||||||
# self-heal anyway).
|
|
||||||
#
|
|
||||||
# Password rotation: update the Secret, delete the machine
|
|
||||||
# (`cscli machines delete crowdsec-agent-workstation`), trigger
|
|
||||||
# janitor-now (recreates it), then `kubectl rollout restart
|
|
||||||
# ds/crowdsec-agent -n crowdsec` (agent reads the file at startup only).
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ServiceAccount
|
|
||||||
metadata:
|
|
||||||
name: crowdsec-janitor
|
|
||||||
namespace: crowdsec
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/part-of: crowdsec
|
|
||||||
---
|
|
||||||
apiVersion: rbac.authorization.k8s.io/v1
|
|
||||||
kind: Role
|
|
||||||
metadata:
|
|
||||||
name: crowdsec-janitor
|
|
||||||
namespace: crowdsec
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/part-of: crowdsec
|
|
||||||
rules:
|
|
||||||
- apiGroups: [""]
|
|
||||||
resources: ["pods"]
|
|
||||||
verbs: ["get", "list"]
|
|
||||||
- apiGroups: [""]
|
|
||||||
resources: ["pods/exec"]
|
|
||||||
verbs: ["create"]
|
|
||||||
- apiGroups: ["apps"]
|
|
||||||
resources: ["daemonsets"]
|
|
||||||
verbs: ["get", "patch"]
|
|
||||||
# `kubectl exec deploy/<name>` resolves deploy -> replicaset -> pod,
|
|
||||||
# which needs read access to these (exec itself is pods/exec above).
|
|
||||||
- apiGroups: ["apps"]
|
|
||||||
resources: ["deployments", "replicasets"]
|
|
||||||
verbs: ["get", "list"]
|
|
||||||
---
|
|
||||||
apiVersion: rbac.authorization.k8s.io/v1
|
|
||||||
kind: RoleBinding
|
|
||||||
metadata:
|
|
||||||
name: crowdsec-janitor
|
|
||||||
namespace: crowdsec
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/part-of: crowdsec
|
|
||||||
subjects:
|
|
||||||
- kind: ServiceAccount
|
|
||||||
name: crowdsec-janitor
|
|
||||||
namespace: crowdsec
|
|
||||||
roleRef:
|
|
||||||
kind: Role
|
|
||||||
name: crowdsec-janitor
|
|
||||||
apiGroup: rbac.authorization.k8s.io
|
|
||||||
---
|
|
||||||
apiVersion: batch/v1
|
|
||||||
kind: CronJob
|
|
||||||
metadata:
|
|
||||||
name: crowdsec-janitor
|
|
||||||
namespace: crowdsec
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/part-of: crowdsec
|
|
||||||
spec:
|
|
||||||
schedule: "17 * * * *"
|
|
||||||
concurrencyPolicy: Forbid
|
|
||||||
successfulJobsHistoryLimit: 3
|
|
||||||
failedJobsHistoryLimit: 3
|
|
||||||
jobTemplate:
|
|
||||||
spec:
|
|
||||||
activeDeadlineSeconds: 300
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/part-of: crowdsec
|
|
||||||
spec:
|
|
||||||
serviceAccountName: crowdsec-janitor
|
|
||||||
restartPolicy: OnFailure
|
|
||||||
containers:
|
|
||||||
- name: janitor
|
|
||||||
# Same image the chart itself uses for registration jobs;
|
|
||||||
# IfNotPresent so it works while the node is offline
|
|
||||||
# (layer cached from the chart install).
|
|
||||||
image: alpine/kubectl:latest
|
|
||||||
imagePullPolicy: IfNotPresent
|
|
||||||
env:
|
|
||||||
- name: AGENT_PASSWORD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: crowdsec-agent-credentials
|
|
||||||
key: password
|
|
||||||
command:
|
|
||||||
- /bin/sh
|
|
||||||
- -c
|
|
||||||
- |
|
|
||||||
set -eu
|
|
||||||
LAPI_EXEC="kubectl exec -n crowdsec deploy/crowdsec-lapi --"
|
|
||||||
echo "== 1. enforce patched agent init =="
|
|
||||||
CUR=$(kubectl get ds crowdsec-agent -n crowdsec \
|
|
||||||
-o jsonpath='{.spec.template.spec.initContainers[0].command[2]}')
|
|
||||||
case "$CUR" in
|
|
||||||
*'-s /tmp_config'*)
|
|
||||||
echo "init already patched"
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
echo "patching init"
|
|
||||||
WAIT='until nc "$LAPI_HOST" "$LAPI_PORT" -z'
|
|
||||||
WAIT="$WAIT; do echo waiting for lapi to start; sleep 5; done"
|
|
||||||
LINK='ln -s /staging/etc/crowdsec /etc/crowdsec'
|
|
||||||
REG='cscli lapi register --machine "$USERNAME"'
|
|
||||||
REG="$REG -u \"\$LAPI_URL\" --token \"\$REGISTRATION_TOKEN\""
|
|
||||||
CREDS=/tmp_config/local_api_credentials.yaml
|
|
||||||
CMD="$WAIT; $LINK; [ -s $CREDS ] || {"
|
|
||||||
CMD="$CMD $REG && cp"
|
|
||||||
CMD="$CMD /etc/crowdsec/local_api_credentials.yaml $CREDS; }"
|
|
||||||
ESC=$(printf '%s' "$CMD" | sed 's/"/\\"/g')
|
|
||||||
PATCH='{"spec":{"template":{"spec":{"initContainers":'
|
|
||||||
PATCH=$PATCH'[{"name":"wait-for-lapi-and-register",'
|
|
||||||
PATCH=$PATCH'"command":["sh","-c","'$ESC'"]}]}}}}'
|
|
||||||
kubectl patch ds crowdsec-agent -n crowdsec \
|
|
||||||
--type strategic -p "$PATCH"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
echo "== 2. prune orphan machines (no heartbeat for 2h) =="
|
|
||||||
$LAPI_EXEC cscli machines prune --duration 2h --force
|
|
||||||
echo "== 3. ensure static machine exists =="
|
|
||||||
if $LAPI_EXEC cscli machines inspect \
|
|
||||||
crowdsec-agent-workstation >/dev/null 2>&1; then
|
|
||||||
echo "static machine present"
|
|
||||||
else
|
|
||||||
echo "recreating static machine"
|
|
||||||
$LAPI_EXEC cscli machines add crowdsec-agent-workstation \
|
|
||||||
--password "$AGENT_PASSWORD" --force
|
|
||||||
fi
|
|
||||||
echo "== 4. prune stale bouncers (no pull for 30d) =="
|
|
||||||
$LAPI_EXEC cscli bouncers prune -d 720h --force
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: crowdsec
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/part-of: crowdsec
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: NetworkPolicy
|
|
||||||
metadata:
|
|
||||||
name: crowdsec-lapi
|
|
||||||
namespace: crowdsec
|
|
||||||
spec:
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
k8s-app: crowdsec
|
|
||||||
type: lapi
|
|
||||||
policyTypes:
|
|
||||||
- Ingress
|
|
||||||
ingress:
|
|
||||||
- from:
|
|
||||||
- namespaceSelector:
|
|
||||||
matchLabels:
|
|
||||||
kubernetes.io/metadata.name: traefik
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: traefik
|
|
||||||
- podSelector:
|
|
||||||
matchLabels:
|
|
||||||
k8s-app: crowdsec
|
|
||||||
type: agent
|
|
||||||
ports:
|
|
||||||
- protocol: TCP
|
|
||||||
port: 8080
|
|
||||||
- from:
|
|
||||||
- namespaceSelector:
|
|
||||||
matchLabels:
|
|
||||||
kubernetes.io/metadata.name: prometheus
|
|
||||||
ports:
|
|
||||||
- protocol: TCP
|
|
||||||
port: 6060
|
|
||||||
+50
-42
@@ -1,46 +1,54 @@
|
|||||||
services:
|
services:
|
||||||
dockmon:
|
dockmon:
|
||||||
image: darthnorse/dockmon:2.5.0
|
image: darthnorse/dockmon:latest
|
||||||
container_name: dockmon
|
container_name: dockmon
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
# ports:
|
ports:
|
||||||
# - 8000:443
|
- 8000:443
|
||||||
volumes:
|
environment:
|
||||||
- data:/app/data
|
- TZ=Europe/Bratislava
|
||||||
- /var/run/docker.sock:/var/run/docker.sock
|
volumes:
|
||||||
healthcheck:
|
- ./data:/app/data
|
||||||
test: ["CMD", "curl", "-k", "-f", "https://localhost:443/health"]
|
- /var/run/docker.sock:/var/run/docker.sock
|
||||||
interval: 30s
|
healthcheck:
|
||||||
timeout: 10s
|
test: [ "CMD", "curl", "-k", "-f", "https://localhost:443/health" ]
|
||||||
retries: 3
|
interval: 30s
|
||||||
labels:
|
timeout: 10s
|
||||||
- "traefik.enable=true"
|
retries: 3
|
||||||
- "traefik.http.services.dockmon.loadbalancer.server.port=443"
|
networks:
|
||||||
- "traefik.http.services.dockmon.loadbalancer.server.scheme=https"
|
- traefik-proxy
|
||||||
- "traefik.http.services.dockmon.loadbalancer.serverstransport=insecureTransport@file"
|
labels:
|
||||||
|
- "traefik.enable=true"
|
||||||
|
- "traefik.docker.network=traefik-proxy"
|
||||||
|
|
||||||
# Prod Router
|
# Prod Router
|
||||||
- "traefik.http.routers.dockmon.rule=Host(`dockmon.forust.xyz`)"
|
- "traefik.http.routers.dockmon.rule=Host(`dockmon.forust.xyz`)"
|
||||||
- "traefik.http.routers.dockmon.entrypoints=websecure"
|
- "traefik.http.routers.dockmon.entrypoints=websecure"
|
||||||
- "traefik.http.routers.dockmon.middlewares=security-headers@file"
|
- "traefik.http.routers.dockmon.middlewares=security-chain@file"
|
||||||
- "traefik.http.routers.dockmon.tls.certresolver=letsencrypt"
|
- "traefik.http.routers.dockmon.service=dockmon"
|
||||||
# Local Router
|
- "traefik.http.routers.dockmon.tls=true"
|
||||||
- "traefik.http.routers.dockmon-local.rule=Host(`dockmon.workstation.internal`)"
|
- "traefik.http.services.dockmon.loadbalancer.server.port=443"
|
||||||
- "traefik.http.routers.dockmon-local.entrypoints=websecure"
|
- "traefik.http.services.dockmon.loadbalancer.server.scheme=https"
|
||||||
- "traefik.http.routers.dockmon-local.tls=true"
|
- "traefik.http.services.dockmon.loadbalancer.serverstransport=insecureTransport@file"
|
||||||
# Dev Router
|
|
||||||
- "traefik.http.routers.dockmon-dev.rule=Host(`dockmon.gigaforust.internal`)"
|
# Local Router
|
||||||
- "traefik.http.routers.dockmon-dev.entrypoints=websecure"
|
- "traefik.http.routers.dockmon-local.rule=Host(`dockmon.workstation.internal`) || Host(`dockmon.internal`)"
|
||||||
- "traefik.http.routers.dockmon-dev.tls=true"
|
- "traefik.http.routers.dockmon-local.entrypoints=websecure"
|
||||||
|
- "traefik.http.routers.dockmon-local.middlewares=security-headers@file"
|
||||||
|
- "traefik.http.routers.dockmon-local.service=dockmon"
|
||||||
|
- "traefik.http.routers.dockmon-local.tls=true"
|
||||||
|
|
||||||
|
# Dev Router
|
||||||
|
- "traefik.http.routers.dockmon-dev.rule=Host(`dockmon.gigaforust.internal`)"
|
||||||
|
- "traefik.http.routers.dockmon-dev.entrypoints=websecure"
|
||||||
|
- "traefik.http.routers.dockmon-dev.middlewares=security-chain@file"
|
||||||
|
- "traefik.http.routers.dockmon-dev.service=dockmon"
|
||||||
|
- "traefik.http.routers.dockmon-dev.tls=true"
|
||||||
|
|
||||||
|
- glance.name=dockmon
|
||||||
|
- glance.url=https://dockmon.forust.xyz/
|
||||||
|
- glance.description=Dockmon is a lightweight Docker container monitoring and management tool with a user-friendly web interface.
|
||||||
|
|
||||||
# Glance Metadata
|
|
||||||
- glance.name=dockmon
|
|
||||||
- glance.url=https://dockmon.forust.xyz/
|
|
||||||
- glance.description=Dockmon is a lightweight Docker container monitoring and management tool with a user-friendly web interface.
|
|
||||||
networks:
|
|
||||||
- proxy
|
|
||||||
volumes:
|
|
||||||
data:
|
|
||||||
networks:
|
networks:
|
||||||
proxy:
|
traefik-proxy:
|
||||||
external: true
|
external: true
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: dockmon-prod-tls
|
|
||||||
namespace: dockmon
|
|
||||||
spec:
|
|
||||||
secretName: dockmon-prod-tls
|
|
||||||
dnsNames:
|
|
||||||
- dockmon.forust.xyz
|
|
||||||
issuerRef:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
kind: ClusterIssuer
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: internal-wildcard-tls
|
|
||||||
namespace: dockmon
|
|
||||||
spec:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
dnsNames:
|
|
||||||
- "*.workstation.internal"
|
|
||||||
- "*.gigaforust.internal"
|
|
||||||
- workstation.internal
|
|
||||||
- gigaforust.internal
|
|
||||||
issuerRef:
|
|
||||||
name: internal-ca
|
|
||||||
kind: ClusterIssuer
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: dockmon-service
|
|
||||||
namespace: dockmon
|
|
||||||
spec:
|
|
||||||
clusterIP: None
|
|
||||||
selector:
|
|
||||||
app: dockmon
|
|
||||||
ports:
|
|
||||||
- port: 443
|
|
||||||
targetPort: 443
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: StatefulSet
|
|
||||||
metadata:
|
|
||||||
name: dockmon-statefulset
|
|
||||||
namespace: dockmon
|
|
||||||
spec:
|
|
||||||
serviceName: dockmon-service
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: dockmon
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: dockmon
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: dockmon
|
|
||||||
image: darthnorse/dockmon:2.5.0
|
|
||||||
ports:
|
|
||||||
- containerPort: 443
|
|
||||||
volumeMounts:
|
|
||||||
- name: data
|
|
||||||
mountPath: /app/data
|
|
||||||
- name: docker-sock
|
|
||||||
mountPath: /var/run/docker.sock
|
|
||||||
livenessProbe:
|
|
||||||
httpGet:
|
|
||||||
path: /health
|
|
||||||
port: 443
|
|
||||||
scheme: HTTPS
|
|
||||||
initialDelaySeconds: 30
|
|
||||||
periodSeconds: 30
|
|
||||||
timeoutSeconds: 10
|
|
||||||
failureThreshold: 3
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "512Mi"
|
|
||||||
cpu: "200m"
|
|
||||||
limits:
|
|
||||||
memory: "1.5Gi"
|
|
||||||
cpu: "700m "
|
|
||||||
volumes:
|
|
||||||
- name: docker-sock
|
|
||||||
hostPath:
|
|
||||||
path: /var/run/docker.sock
|
|
||||||
type: Socket
|
|
||||||
volumeClaimTemplates:
|
|
||||||
- metadata:
|
|
||||||
name: data
|
|
||||||
spec:
|
|
||||||
accessModes: ["ReadWriteOnce"]
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 1Gi
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: ServersTransport
|
|
||||||
metadata:
|
|
||||||
name: dockmon-transport
|
|
||||||
namespace: dockmon
|
|
||||||
spec:
|
|
||||||
insecureSkipVerify: true
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: dockmon-prod
|
|
||||||
namespace: dockmon
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`dockmon.forust.xyz`)
|
|
||||||
kind: Rule
|
|
||||||
middlewares:
|
|
||||||
- name: security-headers@file
|
|
||||||
services:
|
|
||||||
- name: dockmon-service
|
|
||||||
port: 443
|
|
||||||
serversTransport: dockmon-transport
|
|
||||||
tls:
|
|
||||||
secretName: dockmon-prod-tls
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: dockmon-local
|
|
||||||
namespace: dockmon
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`dockmon.workstation.internal`) || Host(`dockmon.gigaforust.internal`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: dockmon-service
|
|
||||||
port: 443
|
|
||||||
serversTransport: dockmon-transport
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: dockmon
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
services:
|
|
||||||
downtify:
|
|
||||||
container_name: downtify
|
|
||||||
image: ghcr.io/henriquesebastiao/downtify:3.4.0
|
|
||||||
restart: unless-stopped
|
|
||||||
# ports:
|
|
||||||
# - '7077:8000'
|
|
||||||
volumes:
|
|
||||||
- ./Downtify_downloads:/downloads
|
|
||||||
labels:
|
|
||||||
- "traefik.enable=true"
|
|
||||||
- "traefik.http.services.downtify.loadbalancer.server.port=8000"
|
|
||||||
|
|
||||||
# Prod Router
|
|
||||||
- "traefik.http.routers.downtify.rule=Host(`downtify.forust.xyz`)"
|
|
||||||
- "traefik.http.routers.downtify.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.downtify.middlewares=security-chain@file"
|
|
||||||
- "traefik.http.routers.downtify.tls.certresolver=letsencrypt"
|
|
||||||
# Local Router
|
|
||||||
- "traefik.http.routers.downtify-local.rule=Host(`downtify.workstation.internal`)"
|
|
||||||
- "traefik.http.routers.downtify-local.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.downtify-local.tls=true"
|
|
||||||
# Dev Router
|
|
||||||
- "traefik.http.routers.downtify-dev.rule=Host(`downtify.gigaforust.internal`)"
|
|
||||||
- "traefik.http.routers.downtify-dev.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.downtify-dev.tls=true"
|
|
||||||
networks:
|
|
||||||
- proxy
|
|
||||||
networks:
|
|
||||||
proxy:
|
|
||||||
external: true
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: downtify-service
|
|
||||||
namespace: downtify
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app: downtify
|
|
||||||
ports:
|
|
||||||
- port: 8000
|
|
||||||
targetPort: 8000
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: downtify-deployment
|
|
||||||
namespace: downtify
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: downtify
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app: downtify
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: downtify
|
|
||||||
image: ghcr.io/henriquesebastiao/downtify:3.4.0
|
|
||||||
ports:
|
|
||||||
- containerPort: 8000
|
|
||||||
volumeMounts:
|
|
||||||
- name: downloads
|
|
||||||
mountPath: /downloads
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
memory: "128Mi"
|
|
||||||
cpu: "200m"
|
|
||||||
limits:
|
|
||||||
memory: "1Gi"
|
|
||||||
cpu: "1"
|
|
||||||
volumes:
|
|
||||||
- name: downloads
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: downtify-downloads-pvc
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: downtify-downloads-pvc
|
|
||||||
namespace: downtify
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 10Gi
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: downtify-prod
|
|
||||||
namespace: downtify
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`downtify.forust.xyz`)
|
|
||||||
kind: Rule
|
|
||||||
middlewares:
|
|
||||||
- name: security-chain@file
|
|
||||||
services:
|
|
||||||
- name: downtify-service
|
|
||||||
port: 8000
|
|
||||||
tls:
|
|
||||||
secretName: downtify-prod-tls
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: downtify-local
|
|
||||||
namespace: downtify
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`downtify.workstation.internal`) || Host(`downtify.gigaforust.internal`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: downtify-service
|
|
||||||
port: 8000
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Namespace
|
|
||||||
metadata:
|
|
||||||
name: downtify
|
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
FROM python:3.9-alpine
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Установка зависимостей
|
||||||
|
COPY requirements.txt .
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
# Копирование кода
|
||||||
|
COPY main.py .
|
||||||
|
COPY .env .
|
||||||
|
# Запуск бота
|
||||||
|
CMD ["python", "-u", "main.py"]
|
||||||
Loaded 100 of 499 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user