forust/homelab
0
0 Fork 0
Code Issues 1 Pull Requests 3 Actions Packages Wiki Activity
Files
a5d384a4d8babdaccdb09f34f1661348f8794ee9
homelab/.gitea/workflows/ci.yaml
T
forustandClaude Opus 4.8 baedea504d ci: scan dependencies for new advisories and stop handing out a write token 
Two things, both about not finding out late.

No workflow declared `permissions`, so all eighteen jobs across the four
workflows ran on a token with the default full repository scope. Every one of
them only checks out code, and deploy reaches the cluster over SSH with the
deploy key, and Renovate writes through its own bot PAT rather than the
Actions token. So `contents: read` is all any of them needed.

The panel image ships 15 known advisories and nothing was looking. Add a
scan-deps job that fails on anything new, and record the eight current ones by
ID in the workflow. It is a list rather than a baseline count so that the diff
that accepts an advisory says so in words, and it lives in our workflow
instead of the package manifest so a subtree sync from forust/userbot cannot
quietly widen the exemption.

Both halves were checked to fail on a regression, not just to pass today:
removing one --ignore-vuln turns the Python step red, and dropping
--audit-level to moderate turns the npm one red on the devalue advisory.

npm audits production dependencies only. All seven findings in the full tree
are build- or test-time: the esbuild advisory needs a vite dev server exposed
to the internet, and nanoid's infinite loop needs a custom generator called
with size 0, which postcss does not do. None are in the 91 kB bundle the panel
serves, so failing on them would be noise that trains people to ignore the
job.

The starlette entries are the reason the job is not "fail on everything":
fastapi 0.115.12 pins starlette<0.47.0 and the last four fixes need 0.49.1
through 1.3.1, so clearing them is a jump to fastapi 0.141.x and is upstream's
call, not a drive-by. Four of the seven are reachable in principle, which the
comment on the job sets out. The panel answers only on
userbot.workstation.internal with no public route, which is what keeps those
four from being an internet-facing DoS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-27 10:46:01 +02:00

678 lines
25 KiB
YAML
Raw Blame History

name: ci
on:
push:
branches:
- "**"
pull_request:
workflow_dispatch:
# Every job here is checkout plus local tools. The token needs to read the tree
# and nothing else, and saying so keeps a future step that reaches for the API
# from quietly holding a token that can write to the repository.
permissions:
contents: read
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
env:
REGISTRY: gcr.forust.xyz
jobs:
lint-compose:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
# Structure check for every committed Compose file, active or not.
# Interpolation, env-file and bind-mount resolution are all switched off,
# because inactive stacks have no .env here and would only fail on their
# ${VAR:?} guards. Active stacks get the full check with interpolation in
# the deploy workflow, where the real .env files live.
- name: Validate Compose files
shell: bash
run: |
set -euo pipefail
source .gitea/workflows/compose-lint.sh
mapfile -t safe_flags < <(compose_safe_flags)
echo "docker compose config ${safe_flags[*]-}"
mapfile -t files < <(compose_files)
if [ "${#files[@]}" -eq 0 ]; then
echo "No Compose files found."
exit 0
fi
failed=0
for f in "${files[@]}"; do
if ! out="$(validate_compose_file "$f" ${safe_flags[@]+"${safe_flags[@]}"} 2>&1)"; then
failed=1
echo "::error file=${f}::$(printf '%s' "$out" | head -1)"
fi
done
if [ "$failed" -ne 0 ]; then
echo "Compose validation failed."
exit 1
fi
echo "checked ${#files[@]} Compose file(s)"
lint-actionlint:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Lint Gitea Actions workflows with actionlint
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh actionlint)"
export PATH="$tools_dir:$PATH"
actionlint -config-file .gitea/actionlint.yaml -color .gitea/workflows/*.yaml
lint-shellcheck:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Lint shell scripts with ShellCheck
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh shellcheck)"
export PATH="$tools_dir:$PATH"
# userbot/ is a git subtree synced from forust/userbot, so its shell
# scripts are upstream's to maintain, not ours. Linting them would let a
# routine subtree pull turn the deploy gate red on code we do not own.
mapfile -t scripts < <(
git ls-files '*.sh' ':(glob)**/*.bash' ':!userbot/**'
)
if [ "${#scripts[@]}" -eq 0 ]; then
echo "No shell scripts found."
exit 0
fi
shellcheck --external-sources --source-path=SCRIPTDIR --severity=style "${scripts[@]}"
lint-prettier:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Check formatting with Prettier
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh prettier)"
export PATH="$tools_dir:$PATH"
mapfile -t prettier_files < <(
git ls-files \
| grep -E '\.(md|json|ya?ml|html|css)$' \
| grep -Ev '^(\.docs/|\.zed/|errorpages/html/|homepages/(forust_files|xdfnx_files)/)'
)
if [ "${#prettier_files[@]}" -eq 0 ]; then
echo "No Prettier-managed files found."
exit 0
fi
prettier --check --ignore-unknown "${prettier_files[@]}"
lint-ruff:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Lint and format-check Python with Ruff
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh ruff)"
export PATH="$tools_dir:$PATH"
ruff check .
ruff format --check .
lint-yaml:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Lint YAML syntax
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh yamllint)"
export PATH="$tools_dir:$PATH"
mapfile -t yaml_files < <(
git ls-files '*.yaml' '*.yml' \
':!node_modules/**' \
':!**/.venv/**'
)
if [ "${#yaml_files[@]}" -eq 0 ]; then
echo "No YAML files found."
exit 0
fi
yamllint -c .yamllint "${yaml_files[@]}"
lint-dockerfiles:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Lint Dockerfiles
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh hadolint)"
export PATH="$tools_dir:$PATH"
mapfile -t dockerfiles < <(
git ls-files ':(glob)**/Dockerfile' ':(glob)**/Dockerfile.*'
)
if [ "${#dockerfiles[@]}" -eq 0 ]; then
echo "No Dockerfiles found."
exit 0
fi
hadolint -c .hadolint.yaml "${dockerfiles[@]}"
# Known, accepted, and recorded. Each line is a real advisory against a
# package we build into the panel image, kept in this workflow rather than in
# the package manifest so that a subtree sync from forust/userbot cannot
# silently widen the exemption.
#
# starlette is the reason this job is not simply "fail on everything":
# fastapi 0.115.12 pins `starlette<0.47.0`, and the fixes for the last four
# below need 0.49.1 through 1.3.1, so clearing them means a jump from fastapi
# 0.115.12 to 0.141.x. That is upstream's call, not a drive-by in a lint
# commit. Of the seven, four are reachable here in principle: 1942 is a
# crafted Range header hitting FileResponse, and the panel serves its built
# SPA through exactly that; 249 is request.form() ignoring max_fields for
# x-www-form-urlencoded, which is the login form; 1941 is a large multipart
# body blocking the event loop; 161 and 248 are unvalidated Host and request
# path reaching request.url. 2280 needs HTTPEndpoint, which the panel does
# not use, and 2281 is Windows-only, and this deploys on Linux.
#
# The panel answers on userbot.workstation.internal and has no public
# forust.xyz route, which is what keeps the four reachable ones from being
# an internet-facing DoS. It still manages Telegram credentials.
#
# Deleting an entry here is how you accept a new advisory, so the diff says
# so out loud.
scan-deps:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Audit the Python dependencies that ship in the image
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh pip-audit)"
export PATH="$tools_dir:$PATH"
# requirements.txt, not requirements-dev.txt: this is what the image
# installs, and the test tooling is not a shipped attack surface.
pip-audit -r userbot/panel/backend/requirements.txt --strict \
--ignore-vuln CVE-2025-67720 \
--ignore-vuln PYSEC-2026-161 \
--ignore-vuln PYSEC-2026-1941 \
--ignore-vuln PYSEC-2026-1942 \
--ignore-vuln PYSEC-2026-2280 \
--ignore-vuln PYSEC-2026-2281 \
--ignore-vuln PYSEC-2026-248 \
--ignore-vuln PYSEC-2026-249
# devDependencies are excluded on purpose. `npm audit` on the full tree
# reports 7 findings, and every one of them is a build- or test-time
# package: the esbuild CORS advisory needs a vite dev server serving to
# the internet, and nanoid's infinite loop needs a custom generator
# called with size 0, which postcss does not do. None of them are in the
# 91 kB bundle the panel serves. The one production finding, devalue
# via svelte, is moderate, which is where --audit-level draws the line;
# this fails on the next high or critical one.
- name: Audit the production npm dependencies
shell: bash
run: |
set -euo pipefail
cd userbot/panel/frontend
npm ci
npm audit --omit=dev --audit-level=high
test-backend:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
# 25 tests over the panel's pydantic models, its auth flow, the SPA
# fallback and the Kubernetes client it shells out with. They existed and
# had never been executed by anything.
#
# Note that userbot/ is a subtree synced from forust/userbot, so a routine
# sync can turn this red on upstream's code. Unlike the shellcheck job,
# which skips that tree because style disagreements there are ours to
# lose, a failing test here is a real defect in a service we deploy.
- name: Run the panel backend test suite
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh uv)"
export PATH="$tools_dir:$PATH"
# A venv in a temp dir rather than a checked-out one: the runner is
# shared, and a leftover .venv would let a dependency the
# requirements no longer pin still satisfy an import.
venv="$(mktemp -d)/venv"
uv venv --quiet "$venv"
uv pip install --quiet --python "$venv/bin/python" \
-r userbot/panel/backend/requirements-dev.txt
# `python -m`, not bare `pytest`: the tests import `app.*` relative to
# the backend directory, which only works if the cwd is on sys.path,
# and only `python -m` puts it there.
cd userbot/panel/backend
"$venv/bin/python" -m pytest tests/ -q
test-frontend:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
# One `npm ci` for both checks below: it is by far the slowest part of
# this job, and a second one would learn nothing the first did not.
#
# `npm ci`, not `npm install`, for the same reason the Dockerfile uses it:
# the lockfile is what makes the tree that gets checked the tree that
# gets shipped.
- name: Type-check and test the panel frontend
shell: bash
run: |
set -euo pipefail
cd userbot/panel/frontend
npm ci
# svelte-check has been a devDependency all along with no script
# pointing at it, so the type errors it reports had nowhere to
# surface. It is clean today, which is the only reason it can be a
# gate: it stops at whatever upstream introduces rather than
# reporting a backlog we inherited.
npm run check
npm test
validate:
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 20
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Validate Kubernetes manifests against JSON schemas
shell: bash
run: |
set -euo pipefail
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh kubeconform)"
export PATH="$tools_dir:$PATH"
mapfile -t manifests < <(
git ls-files ':(glob)**/k8s/**/*.yaml' ':(glob)**/k8s/**/*.yml' \
| grep -Ev '(^|/)(kustomization\.ya?ml|.*\.example\.ya?ml|.*values\.ya?ml|patch-.*\.ya?ml)$'
)
if [ "${#manifests[@]}" -eq 0 ]; then
echo "No Kubernetes manifests found."
exit 0
fi
kubeconform \
-strict \
-ignore-missing-schemas \
-summary \
"${manifests[@]}"
# kubeconform has no schemas for CRDs, so every IngressRoute, Certificate,
# PrometheusRule, Middleware, ServersTransport and ServiceMonitor is silently
# skipped above. The live API server knows the real CRD schemas (and runs the
# cert-manager / Traefik admission webhooks), so validate there too.
#
# Only services marked with a k8s/active marker are checked: server-side
# dry-run needs the target namespace to exist, and inactive services are not
# deployed. Services being enabled for the first time are still covered by
# the JSON-schema pass above.
#
# Main pushes only. `--dry-run=server` persists nothing, but it does execute
# the admission webhooks of the production API server, so anyone able to open
# a pull request would be able to run arbitrary manifest content through
# cert-manager and Traefik. A pull request has nothing to gain from it either:
# only main is ever deployed, and this job runs to completion before the
# deploy workflow is allowed to start, so a bad CRD is still caught before
# anything reaches the cluster -- just on the push rather than on the PR.
- name: Note the server-side check is not running here
if: github.event_name == 'pull_request' || github.ref != 'refs/heads/main'
shell: bash
run: |
echo "::notice::Skipping the server-side dry-run. It executes the cert-manager and" \
"Traefik admission webhooks against the production API server, so it is limited" \
"to pushes to main. CRDs are still schema-checked by kubeconform above, and the" \
"server-side pass still runs on main before the deploy."
- name: Validate active manifests against the live API server
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
shell: bash
run: |
set -euo pipefail
if ! kubectl get --raw='/readyz' --request-timeout=10s >/dev/null 2>&1; then
echo "::warning::Cluster unreachable — skipped server-side validation of CRDs (IngressRoute, Certificate, PrometheusRule). Review manifest changes manually."
exit 0
fi
mapfile -t k8s_dirs < <(
git ls-files '*.yaml' '*.yml' \
| grep -E '(^|/)k8s/' \
| sed -E 's#((^|.*/)k8s)/.*#\1#' \
| sort -u
)
manifests=()
kustomize_apps=()
for dir in "${k8s_dirs[@]}"; do
if [ ! -f "${dir}/active" ]; then
echo "skip (no k8s/active): ${dir}"
continue
fi
if [ -f "${dir}/overlays/prod/kustomization.yaml" ]; then
kustomize_apps+=("${dir}/overlays/prod")
elif [ -f "${dir}/base/kustomization.yaml" ]; then
kustomize_apps+=("${dir}/base")
else
while IFS= read -r f; do
[ -n "$f" ] && manifests+=("$f")
done < <(
git ls-files "${dir}/*.yaml" "${dir}/*.yml" \
| grep -Ev '(^|/)(kustomization\.ya?ml|.*\.example\.ya?ml|.*values\.ya?ml|patch-.*\.ya?ml)$'
)
fi
done
echo "server-side dry-run: ${#manifests[@]} manifests, ${#kustomize_apps[@]} kustomize apps"
failed=0
for m in ${manifests[@]+"${manifests[@]}"}; do
if ! out="$(kubectl apply --dry-run=server -f "$m" 2>&1)"; then
failed=1
echo "::error file=${m}::$(printf '%s' "$out" | head -1)"
fi
done
for k in ${kustomize_apps[@]+"${kustomize_apps[@]}"}; do
if ! out="$(kubectl apply -k "$k" --dry-run=server 2>&1)"; then
failed=1
echo "::error file=${k}::$(printf '%s' "$out" | head -1)"
fi
done
if [ "$failed" -ne 0 ]; then
echo "Server-side validation failed. The API server (or an admission webhook) rejected these manifests."
exit 1
fi
echo "server-side dry-run: all active manifests accepted by the API server"
build:
needs:
[lint-actionlint, lint-shellcheck, lint-compose, lint-prettier, lint-ruff, lint-yaml, lint-dockerfiles, validate]
if: github.event_name != 'pull_request' && (github.ref_name == 'main' || github.ref_name == 'dev')
runs-on: [self-hosted, linux, arch, homelab]
timeout-minutes: 60
outputs:
services: ${{ steps.services.outputs.services }}
steps:
- name: Checkout repository
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
fetch-depth: 0
- name: Detect changed docker-built services
id: services
shell: bash
run: |
base="${{ github.event.before }}"
if [ -z "$base" ] || [ "$base" = "0000000000000000000000000000000000000000" ]; then
base="$(git rev-list --max-parents=0 HEAD)"
fi
mapfile -t changed_files < <(git diff --name-only "$base" "${GITHUB_SHA}")
services=()
add_service() {
local name="$1"
local seen=0
for existing in "${services[@]}"; do
if [ "$existing" = "$name" ]; then
seen=1
break
fi
done
if [ "$seen" -eq 0 ]; then
services+=("$name")
fi
}
for file in "${changed_files[@]}"; do
case "$file" in
dtek_notif/*)
add_service dtek_notif
;;
errorpages/*)
add_service errorpages
;;
userbot/*)
add_service userbot
;;
homepages/*)
add_service homepages
;;
edu_master/phpsessid-bot/*|edu_master/webinar-checker/*|edu_master/compose.yaml)
add_service edu_master
;;
esac
done
if [ "${#services[@]}" -eq 0 ]; then
echo "No docker-built services changed."
echo "services=" >> "$GITHUB_OUTPUT"
exit 0
fi
printf '%s\n' "${services[@]}" | tee /tmp/services.txt
echo "services=$(paste -sd, /tmp/services.txt)" >> "$GITHUB_OUTPUT"
- name: Log in to registry
if: steps.services.outputs.services != ''
shell: bash
run: |
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login "${REGISTRY}" \
-u "${{ secrets.REGISTRY_USERNAME }}" \
--password-stdin
- name: Build and push changed images
if: steps.services.outputs.services != ''
shell: bash
run: |
IFS=, read -r -a services <<< "${{ steps.services.outputs.services }}"
for service in "${services[@]}"; do
case "$service" in
dtek_notif)
image="${REGISTRY}/forust/dtek-notif"
tags=()
case "${GITHUB_REF_NAME}" in
main)
tags+=("main" "prod")
;;
dev)
tags+=("dev")
;;
esac
build_args=()
for tag in "${tags[@]}"; do
build_args+=(-t "${image}:${tag}")
done
docker build \
--cache-from "type=registry,ref=${image}:buildcache" \
--cache-to "type=registry,ref=${image}:buildcache,mode=max" \
"${build_args[@]}" dtek_notif
for tag in "${tags[@]}"; do
docker push "${image}:${tag}"
done
;;
errorpages)
image="${REGISTRY}/forust/error-pages"
tags=()
case "${GITHUB_REF_NAME}" in
main)
tags+=("main" "prod")
;;
dev)
tags+=("dev")
;;
esac
build_args=()
for tag in "${tags[@]}"; do
build_args+=(-t "${image}:${tag}")
done
docker build \
--cache-from "type=registry,ref=${image}:buildcache" \
--cache-to "type=registry,ref=${image}:buildcache,mode=max" \
"${build_args[@]}" errorpages
for tag in "${tags[@]}"; do
docker push "${image}:${tag}"
done
;;
userbot)
tags=()
case "${GITHUB_REF_NAME}" in
main)
tags+=("main" "prod")
;;
dev)
tags+=("dev")
;;
esac
for target in runtime panel; do
case "$target" in
runtime)
context="userbot"
image="${REGISTRY}/forust/userbot"
;;
panel)
context="userbot/panel"
image="${REGISTRY}/forust/userbot-panel"
;;
esac
build_args=()
for tag in "${tags[@]}"; do
build_args+=(-t "${image}:${tag}")
done
docker build \
--cache-from "type=registry,ref=${image}:buildcache" \
--cache-to "type=registry,ref=${image}:buildcache,mode=max" \
"${build_args[@]}" "$context"
for tag in "${tags[@]}"; do
docker push "${image}:${tag}"
done
done
;;
homepages)
for variant in forust xdfnx; do
case "$variant" in
forust)
image="${REGISTRY}/forust/forust-homepage"
;;
xdfnx)
image="${REGISTRY}/forust/xdfnx-homepage"
;;
esac
tags=()
case "${GITHUB_REF_NAME}" in
main)
tags+=("main" "prod")
;;
dev)
tags+=("dev")
;;
esac
build_args=()
for tag in "${tags[@]}"; do
build_args+=(-t "${image}:${tag}")
done
docker build \
--cache-from "type=registry,ref=${image}:buildcache" \
--cache-to "type=registry,ref=${image}:buildcache,mode=max" \
"${build_args[@]}" -f "homepages/Dockerfile.${variant}" homepages
for tag in "${tags[@]}"; do
docker push "${image}:${tag}"
done
done
;;
edu_master)
for variant in session-keeper webinar-checker; do
case "$variant" in
session-keeper)
context="edu_master/phpsessid-bot"
image="${REGISTRY}/forust/session-keeper"
;;
webinar-checker)
context="edu_master/webinar-checker"
image="${REGISTRY}/forust/webinar-checker"
;;
esac
tags=()
case "${GITHUB_REF_NAME}" in
main)
tags+=("main" "prod")
;;
dev)
tags+=("dev")
;;
esac
build_args=()
for tag in "${tags[@]}"; do
build_args+=(-t "${image}:${tag}")
done
docker build \
--cache-from "type=registry,ref=${image}:buildcache" \
--cache-to "type=registry,ref=${image}:buildcache,mode=max" \
"${build_args[@]}" "$context"
for tag in "${tags[@]}"; do
docker push "${image}:${tag}"
done
done
;;
esac
done
Reference in new issue View Git Blame Copy Permalink