fix(server): k8s rollout readiness
ci / lint-prettier (push) Failing after 10s
ci / lint-ruff (push) Failing after 4s
ci / lint-yaml (push) Successful in 5s
ci / lint-dockerfiles (push) Successful in 5s
ci / validate (push) Successful in 5s
ci / lint-audit (push) Failing after 49s
ci / publish (push) Has been skipped
ci / lint-prettier (push) Failing after 10s
ci / lint-ruff (push) Failing after 4s
ci / lint-yaml (push) Successful in 5s
ci / lint-dockerfiles (push) Successful in 5s
ci / validate (push) Successful in 5s
ci / lint-audit (push) Failing after 49s
ci / publish (push) Has been skipped
- TRUSTED_HOSTS env: configurable trusted hostnames for proxy-domain access (default stays strict: localhost/loopback/private IP); k8s manifest sets tg.workstation.internal (L-8 follow-up) - Media allowlist +10: mkv/mk3d/heic/tgs/flv/3gp/ogv/asf/wmv/djvu (live disk has .tgs x44, .mkv x2) - Cache buster: app.js?v=4 -> ?v=5 so browsers pick up the new bundle - +6 tests (64 passing); REVIEW.md updated with live-cluster rollout notes
This commit is contained in:
@@ -211,4 +211,15 @@
|
||||
- **SSRF**: юзер-контролируемого фетча URL нет (только MTProto).
|
||||
- **Десериализация**: только JSON, без pickle/yaml.
|
||||
- **Command injection**: нет subprocess/os.system в продакшн-путях.
|
||||
- **Secrets в image**: `.dockerignore` исключает `data/` и `session/`.
|
||||
- **Secrets в image**: `.dockerignore` исключает `data/` и `session/`.
|
||||
---
|
||||
|
||||
### Follow-up (шестой проход — k8s-rollout audit живого кластера)
|
||||
|
||||
- **TRUSTED_HOSTS** (`webui_server.py`, L-8): `_is_trusted_host` отвергал ЛЮБОЙ hostname → через Traefik-домен `tg.workstation.internal` (`k8s/telegram-scraper.yaml:97`) браузерные POST/DELETE + SSE `/api/jobs/*/events` возвращали 403. Добавлен `_TRUSTED_HOSTS_ENV` (parse `TRUSTED_HOSTS` на импорте, нормализация `.strip().lower().rstrip(".")`); проверка после localhost-set. Дефолт строгий: без env результаты идентичны прежним для ВСЕХ входов (regression guard); с env настроенный hostname (любой case, опциональный trailing dot) проходит, остальные — нет. IPv6-with-port (`[::1]:8080`) — out of scope, не тронут.
|
||||
- **Media allowlist** (`_MEDIA_FILE_EXTENSIONS`): +10 суффиксов с комментарием `# extended coverage (animated stickers, matroska, legacy containers)` — `.mkv .mk3d .heic .tgs .flv .3gp .ogv .asf .wmv .djvu`. Живой диск: `.tgs` x44 / `.mkv` x2 (возвращали 403). `.exe`/`.ts` оставлены 403. `guess_media_kind` для новых суффиксов возвращает `"file"` (fall-through как у `.zip`) — viewer рендерит "Open file" link. `serve_media` уже lowercases suffix, `.MP4`/`.MOV` ок.
|
||||
- **Cache-buster**: `webui/index.html` `app.js?v=4` → `app.js?v=5` (иначе stale JS после deploy).
|
||||
- **k8s-манифест**: `containers[0].env: TRUSTED_HOSTS="tg.workstation.internal"` (после `tty: true`); image/probes/securityContext/resources/PVCs/IngressRoute не тронуты; YAML проверен `yaml.safe_load`.
|
||||
- **Live-cluster факты**: local-path PVCs, state v2 `accounts=[default, forust]`, смешанное владение 1000/root → обязательный `chown -R 1000:1000 /app/data /app/session` ДО первого старта нового пода.
|
||||
|
||||
Тесты: **58 → 64 passed** (+3 TRUSTED_HOSTS env, +3 extended media).
|
||||
|
||||
@@ -41,6 +41,9 @@ spec:
|
||||
image: gcr.forust.xyz/forust/telegram-scraper:latest
|
||||
stdin: true
|
||||
tty: true
|
||||
env:
|
||||
- name: TRUSTED_HOSTS
|
||||
value: "tg.workstation.internal"
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
resources:
|
||||
|
||||
@@ -977,6 +977,103 @@ class TestTrustedHost:
|
||||
assert h3._check_same_origin() is True
|
||||
|
||||
|
||||
class TestTrustedHostsEnv:
|
||||
"""TRUSTED_HOSTS env: proxy-domain fix with strict default.
|
||||
|
||||
_TRUSTED_HOSTS_ENV is read at import, so tests patch the module
|
||||
attribute directly (monkeypatch) instead of mutating os.environ.
|
||||
"""
|
||||
|
||||
def _ws(self):
|
||||
import webui_server as ws_module
|
||||
return ws_module
|
||||
|
||||
def test_default_env_domain_untrusted(self, monkeypatch):
|
||||
ws = self._ws()
|
||||
monkeypatch.setattr(ws, "_TRUSTED_HOSTS_ENV", frozenset())
|
||||
assert ws._is_trusted_host("tg.workstation.internal") is False
|
||||
# regression guard: loopback/private literals still trusted
|
||||
for host in ("localhost", "127.0.0.1", "10.0.0.5", "192.168.1.50"):
|
||||
assert ws._is_trusted_host(host) is True, f"{host!r} should be trusted"
|
||||
|
||||
def test_configured_domain_trusted(self, monkeypatch):
|
||||
ws = self._ws()
|
||||
monkeypatch.setattr(ws, "_TRUSTED_HOSTS_ENV", {"tg.workstation.internal"})
|
||||
assert ws._is_trusted_host("tg.workstation.internal") is True
|
||||
assert ws._is_trusted_host("TG.WORKSTATION.INTERNAL") is True
|
||||
assert ws._is_trusted_host("tg.workstation.internal.") is True
|
||||
assert ws._is_trusted_host("unknown.example") is False
|
||||
|
||||
def test_check_same_origin_with_trusted_domain(self, monkeypatch):
|
||||
ws = self._ws()
|
||||
monkeypatch.setattr(ws, "_TRUSTED_HOSTS_ENV", {"tg.workstation.internal"})
|
||||
h = _make_ws_handler(
|
||||
headers={
|
||||
"Host": "tg.workstation.internal",
|
||||
"Origin": "https://tg.workstation.internal",
|
||||
}
|
||||
)
|
||||
assert h._check_same_origin() is True
|
||||
|
||||
|
||||
class TestExtendedMediaServing:
|
||||
"""Extended media allowlist: .tgs/.mkv/... served, .exe/.ts still 403."""
|
||||
|
||||
def _make_full_media_handler(self, relative, data_dir):
|
||||
import webui_server as ws_module
|
||||
self._ws_orig_data = ws_module.DATA_DIR
|
||||
ws_module.DATA_DIR = data_dir
|
||||
handler = object.__new__(ws_module.TelegramScraperRequestHandler)
|
||||
handler.headers = {}
|
||||
handler.rfile = io.BytesIO()
|
||||
handler.wfile = io.BytesIO()
|
||||
handler.path = "/media/" + relative
|
||||
handler.command = "GET"
|
||||
handler.client_address = ("127.0.0.1", 4321)
|
||||
handler.server = MagicMock()
|
||||
handler.send_error_json = MagicMock()
|
||||
handler.send_response = MagicMock()
|
||||
handler.send_header = MagicMock()
|
||||
handler.end_headers = MagicMock()
|
||||
return handler
|
||||
|
||||
def _serve_and_status(self, relative):
|
||||
import webui_server as ws_module
|
||||
h = self._make_full_media_handler(relative, TEST_DATA)
|
||||
try:
|
||||
ws_module.DATA_DIR = TEST_DATA
|
||||
h.serve_media(relative)
|
||||
finally:
|
||||
ws_module.DATA_DIR = self._ws_orig_data
|
||||
if h.send_error_json.called:
|
||||
return int(h.send_error_json.call_args[0][0])
|
||||
return int(h.send_response.call_args[0][0])
|
||||
|
||||
def test_serves_tgs_and_mkv(self):
|
||||
media_dir = TEST_DATA / "ext-media"
|
||||
media_dir.mkdir(parents=True, exist_ok=True)
|
||||
(media_dir / "sticker.tgs").write_bytes(b"\x1f\x8b\x08\x00")
|
||||
(media_dir / "clip.mkv").write_bytes(b"\x1a\x45\xdf\xa3")
|
||||
assert self._serve_and_status("ext-media/sticker.tgs") == 200
|
||||
assert self._serve_and_status("ext-media/clip.mkv") == 200
|
||||
|
||||
def test_guess_media_kind_new_suffixes_fall_through_to_file(self):
|
||||
import webui_server as ws_module
|
||||
for name in (
|
||||
"a.tgs", "a.mkv", "a.mk3d", "a.heic", "a.flv",
|
||||
"a.3gp", "a.ogv", "a.asf", "a.wmv", "a.djvu",
|
||||
):
|
||||
assert ws_module.guess_media_kind(name, None) == "file", name
|
||||
|
||||
def test_exe_and_ts_still_forbidden(self):
|
||||
media_dir = TEST_DATA / "ext-media"
|
||||
media_dir.mkdir(parents=True, exist_ok=True)
|
||||
(media_dir / "evil.exe").write_bytes(b"MZ")
|
||||
(media_dir / "stream.ts").write_bytes(b"\x47" * 188)
|
||||
assert self._serve_and_status("ext-media/evil.exe") == 403
|
||||
assert self._serve_and_status("ext-media/stream.ts") == 403
|
||||
|
||||
|
||||
class TestMediaServingLockdown:
|
||||
"""M-1: /media/ must never serve state.json / *.db / *.session."""
|
||||
|
||||
|
||||
+1
-1
@@ -372,6 +372,6 @@
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script src="/static/app.js?v=4"></script>
|
||||
<script src="/static/app.js?v=5"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -118,15 +118,25 @@ def parse_bool(value: Any, default: bool = False) -> bool:
|
||||
# ── L-8: trusted-host check ──────────────────────────────────────────────
|
||||
import ipaddress # noqa: E402
|
||||
|
||||
# Comma-separated extra trusted hostnames for browser state-mutating
|
||||
# requests when the panel is served through a proxy domain.
|
||||
# Example: TRUSTED_HOSTS="tg.workstation.internal,example.com"
|
||||
_TRUSTED_HOSTS_ENV = frozenset(
|
||||
h.strip().lower().rstrip(".") for h in os.environ.get("TRUSTED_HOSTS", "").split(",") if h.strip()
|
||||
)
|
||||
|
||||
|
||||
def _is_trusted_host(host: str) -> bool:
|
||||
"""Return True if *host* (the ``Host`` header value) is a loopback /
|
||||
private address that this local-only deployment should trust."""
|
||||
hostname = host.split("@")[-1].split(":")[0] # strip auth / port
|
||||
hostname = hostname.strip().lower().rstrip(".")
|
||||
if not hostname:
|
||||
return False
|
||||
if hostname in {"localhost", "127.0.0.1", "::1"}:
|
||||
return True
|
||||
if hostname in _TRUSTED_HOSTS_ENV:
|
||||
return True
|
||||
try:
|
||||
addr = ipaddress.ip_address(hostname)
|
||||
return addr.is_loopback or addr.is_private or addr.is_link_local
|
||||
@@ -157,6 +167,9 @@ _MEDIA_FILE_EXTENSIONS = frozenset({
|
||||
# archives / other Telegram document types
|
||||
".zip", ".rar", ".7z", ".apk", ".epub", ".tar", ".gz", ".bz2", ".xz",
|
||||
".odt", ".ods", ".odp",
|
||||
# extended coverage (animated stickers, matroska, legacy containers)
|
||||
".mkv", ".mk3d", ".heic", ".tgs", ".flv", ".3gp", ".ogv", ".asf",
|
||||
".wmv", ".djvu",
|
||||
})
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user