forust/telegram-scraper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Activity

Merge branch 'dev'

Browse Source
This commit is contained in:
mr-forust
2026-06-27 16:30:36 +02:00
parent f5ef74bda6 3eb6bdf534
commit 9b72c3128e
12 changed files with 2751 additions and 799 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+57 -24
View File
@@ -3,9 +3,9 @@
Telegram scraper on top of Telethon with: Telegram scraper on top of Telethon with:
- CLI workflow for scraping, exports, media recovery, and forwarding - CLI workflow for scraping, exports, media recovery, and forwarding
- lightweight Web UI served by Python - lightweight multi-account Web UI served by Python
- SQLite storage per tracked chat/channel - SQLite storage per tracked chat/channel
- Docker/Compose setup for "run it and leave it on the server" - Docker/Compose / Kubernetes setup for "run it and leave it on the server"
## What It Does ## What It Does
@@ -15,20 +15,31 @@ Telegram scraper on top of Telethon with:
- Exports to CSV and JSON - Exports to CSV and JSON
- Supports continuous scraping - Supports continuous scraping
- Supports forwarding rules - Supports forwarding rules
- Lets you browse tracked channels and local exports in a web panel - Multi-account support — multiple Telegram sessions side by side
- Lets you browse tracked channels and local exports in a web panel with account tabs
## Project Layout ## Project Layout
```text ```text
. .
├── main.py # starts the web server ├── main.py # starts the web server
├── webui_server.py # HTTP server + API ├── webui_server.py # HTTP server + REST API
├── webui/ # plain HTML/CSS/JS frontend ├── webui/ # plain HTML/CSS/JS frontend
│ ├── index.html / app.js # main dashboard
│ ├── viewer.html / viewer.js # message viewer with account selector
│ └── swagger.html / swagger.js # API docs
├── telegram_scraper_with_forwarding.py ├── telegram_scraper_with_forwarding.py
├── app_state.py # multi-account state management
├── scraper_jobs.py # async job runner
├── health.py # health check endpoints
├── data/ ├── data/
│ ├── state.json # shared settings and credentials │ ├── state.json # global state (accounts list)
│ └── <channel_id>/<channel_id>.db # SQLite databases │ └── accounts/<id>/ # per-account data
└── session/ # Telethon session files │ ├── state.json # per-account settings & credentials
│ └── <channel_id>/ # SQLite DB + media
└── session/
├── <account_id>.session # per-account Telethon session
└── ... # (WAL journal mode for concurrency)
``` ```
## Requirements ## Requirements
@@ -97,11 +108,12 @@ http://<server-ip>:8080
The current web panel includes: The current web panel includes:
- tracked channels overview - multi-account support with account tabs
- tracked channels overview per account
- background job queue - background job queue
- scrape/export/media actions - scrape/export/media actions per account
- shared `scrape_media` toggle - shared `scrape_media` toggle
- local message viewer powered by SQLite + media files - local message viewer with account selector
- API docs at `/swagger` and `/openapi.json` - API docs at `/swagger` and `/openapi.json`
- health checks at `/health` and `/health/continuous` - health checks at `/health` and `/health/continuous`
@@ -113,48 +125,67 @@ If you still want the old interactive mode:
python telegram_scraper_with_forwarding.py python telegram_scraper_with_forwarding.py
``` ```
## Docker ## Docker / Kubernetes
Build and run: ### Docker Compose
```bash ```bash
docker compose up -d --build docker compose up -d --build
``` ```
### Kubernetes
```bash
kubectl apply -f k8s/telegram-scraper.yaml
```
Then open: Then open:
```text ```text
http://<server-ip>:8080 http://<server-ip>:8080
``` ```
### Volumes ### Volumes (Docker)
- `./data:/app/data` for databases, exports, and `state.json` - `./data:/app/data` for databases, exports, and `state.json`
- `session:/app/session` for Telethon sessions - `session:/app/session` for Telethon sessions
If you already logged in before with the same compose volume and did not remove it, the session should be reused. If you already logged in before with the same compose volume and did not remove it, the session should be reused.
## Multi-Account
The app supports multiple Telegram accounts side by side. Each account has:
- its own `state.json` under `data/accounts/<id>/`
- its own Telethon session file `session/<id>.session`
- its own channel databases and media
Use the Web UI account tabs to switch between accounts, or create new ones in Settings.
Legacy single-account data is automatically migrated to the `default` account on first startup.
## Shared State ## Shared State
The Web UI and CLI share the same files: The Web UI and CLI share the same files:
- `data/state.json` - `data/state.json` (global — accounts list)
- `session/session.session` - `data/accounts/<id>/state.json` (per-account credentials and settings)
- `session/<id>.session` (per-account Telethon session, WAL journal mode)
That means: That means:
- credentials can stay in `state.json` - credentials persist in state files
- both entry points use the same tracked channels - both entry points use the same tracked channels per account
- both entry points can reuse the same Telegram login session - both entry points can reuse the same Telegram login sessions
## Data Storage ## Data Storage
### SQLite ### SQLite
Each tracked channel gets its own database: Each tracked channel per account gets its own database:
```text ```text
data/<channel_id>/<channel_id>.db data/accounts/<account_id>/<channel_id>/<channel_id>.db
``` ```
Main fields include: Main fields include:
@@ -179,20 +210,22 @@ Main fields include:
Media files are stored in: Media files are stored in:
```text ```text
data/<channel_id>/media/ data/accounts/<account_id>/<channel_id>/media/
``` ```
### Exports ### Exports
Generated into the same channel folder: Generated into the same channel folder:
- `data/<channel_id>/<channel_id>_<username>.csv` - `data/accounts/<account_id>/<channel_id>/<channel_id>_<username>.csv`
- `data/<channel_id>/<channel_id>_<username>.json` - `data/accounts/<account_id>/<channel_id>/<channel_id>_<username>.json`
## Notes ## Notes
- The web server is intentionally simple: plain HTML/CSS/JS, no frontend framework - The web server is intentionally simple: plain HTML/CSS/JS, no frontend framework
- The viewer currently reads local SQLite data, not Telegram export HTML directly - The viewer reads local SQLite data and supports account switching via a dropdown
- Session SQLite files use WAL journal mode to prevent "database is locked" errors when concurrent scraping and jobs run
- The active account tab is persisted in `localStorage` across page reloads
- If dependencies are installed but the Telegram session is missing, the Web UI falls back to read-only status until login is available - If dependencies are installed but the Telegram session is missing, the Web UI falls back to read-only status until login is available
## Disclaimer ## Disclaimer
app_state.py
+226 -18
View File
@@ -1,11 +1,23 @@
import json import json
import logging
import shutil
import threading import threading
import time import time
from copy import deepcopy
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, Optional from typing import Any, Callable, Dict, List, Optional
logger = logging.getLogger(__name__)
DEFAULT_STATE: Dict[str, Any] = { # ── defaults ──────────────────────────────────────────────────────────
GLOBAL_DEFAULTS: Dict[str, Any] = {
"accounts": [],
"version": 2,
}
ACCOUNT_DEFAULTS: Dict[str, Any] = {
"label": "",
"api_id": None, "api_id": None,
"api_hash": None, "api_hash": None,
"channels": {}, "channels": {},
@@ -20,10 +32,15 @@ DEFAULT_STATE: Dict[str, Any] = {
}, },
} }
# ── StateStore (thread-safe JSON) ─────────────────────────────────────
class StateStore: class StateStore:
def __init__(self, path: Path): """Thread-safe JSON state store with TTL cache."""
def __init__(self, path: Path, defaults: Optional[Dict[str, Any]] = None):
self.path = path self.path = path
self.defaults = defaults or {}
self.lock = threading.RLock() self.lock = threading.RLock()
self._cache: Optional[Dict[str, Any]] = None self._cache: Optional[Dict[str, Any]] = None
self._cache_time: float = 0 self._cache_time: float = 0
@@ -35,15 +52,15 @@ class StateStore:
if self._cache is not None and (now - self._cache_time) < self._cache_ttl: if self._cache is not None and (now - self._cache_time) < self._cache_ttl:
return dict(self._cache) return dict(self._cache)
if not self.path.exists(): if not self.path.exists():
result = self._default_state() result = deepcopy(self.defaults)
self._cache = result self._cache = result
self._cache_time = now self._cache_time = now
return result return result
try: try:
with self.path.open("r", encoding="utf-8") as handle: with self.path.open("r", encoding="utf-8") as handle:
state = json.load(handle) state: Dict[str, Any] = json.load(handle)
except (json.JSONDecodeError, OSError): except (json.JSONDecodeError, OSError):
result = self._default_state() result = deepcopy(self.defaults)
self._cache = result self._cache = result
self._cache_time = now self._cache_time = now
return result return result
@@ -56,10 +73,9 @@ class StateStore:
with self.lock: with self.lock:
self.path.parent.mkdir(parents=True, exist_ok=True) self.path.parent.mkdir(parents=True, exist_ok=True)
tmp_path = self.path.with_suffix(self.path.suffix + ".tmp") tmp_path = self.path.with_suffix(self.path.suffix + ".tmp")
merged = self._merge_defaults(state)
with tmp_path.open("w", encoding="utf-8") as handle: with tmp_path.open("w", encoding="utf-8") as handle:
json.dump( json.dump(merged, handle, ensure_ascii=False, indent=2)
self._merge_defaults(state), handle, ensure_ascii=False, indent=2
)
handle.write("\n") handle.write("\n")
tmp_path.replace(self.path) tmp_path.replace(self.path)
self._cache = None self._cache = None
@@ -73,9 +89,7 @@ class StateStore:
def continuous_config(self) -> Dict[str, Any]: def continuous_config(self) -> Dict[str, Any]:
state = self.load() state = self.load()
return dict( return deepcopy(state.get("continuous_scraping") or ACCOUNT_DEFAULTS["continuous_scraping"])
state.get("continuous_scraping") or DEFAULT_STATE["continuous_scraping"]
)
def save_continuous_config(self, config: Dict[str, Any]) -> Dict[str, Any]: def save_continuous_config(self, config: Dict[str, Any]) -> Dict[str, Any]:
def mutate(state: Dict[str, Any]) -> None: def mutate(state: Dict[str, Any]) -> None:
@@ -92,14 +106,208 @@ class StateStore:
return self.update(mutate)["continuous_scraping"] return self.update(mutate)["continuous_scraping"]
def _default_state(self) -> Dict[str, Any]:
return json.loads(json.dumps(DEFAULT_STATE))
def _merge_defaults(self, state: Dict[str, Any]) -> Dict[str, Any]: def _merge_defaults(self, state: Dict[str, Any]) -> Dict[str, Any]:
merged = self._default_state() merged = deepcopy(self.defaults)
for key, value in state.items(): for key, value in state.items():
if key == "continuous_scraping" and isinstance(value, dict): if isinstance(value, dict) and isinstance(merged.get(key), dict):
merged[key].update(value) nested = deepcopy(merged[key])
nested.update(value)
merged[key] = nested
else: else:
merged[key] = value merged[key] = value
return merged return merged
# ── global state helpers ──────────────────────────────────────────────
_GLOBAL_STORE: Optional[StateStore] = None
def get_global_store(data_dir: Path) -> StateStore:
global _GLOBAL_STORE
if _GLOBAL_STORE is None:
_GLOBAL_STORE = StateStore(data_dir / "state.json", defaults=GLOBAL_DEFAULTS)
return _GLOBAL_STORE
def load_global(data_dir: Path) -> Dict[str, Any]:
return get_global_store(data_dir).load()
def save_global(data_dir: Path, state: Dict[str, Any]) -> None:
get_global_store(data_dir).save(state)
def list_accounts(data_dir: Path) -> List[str]:
return list(load_global(data_dir).get("accounts", []))
def account_exists(data_dir: Path, account_id: str) -> bool:
return account_id in list_accounts(data_dir)
# ── per-account state helpers ─────────────────────────────────────────
_ACCOUNT_STORES: Dict[str, StateStore] = {}
_account_stores_lock = threading.Lock()
def get_account_store(data_dir: Path, account_id: str) -> StateStore:
global _ACCOUNT_STORES
with _account_stores_lock:
key = f"{data_dir}:{account_id}"
if key not in _ACCOUNT_STORES:
store = StateStore(
data_dir / "accounts" / account_id / "state.json",
defaults=ACCOUNT_DEFAULTS,
)
_ACCOUNT_STORES[key] = store
return _ACCOUNT_STORES[key]
def load_account(data_dir: Path, account_id: str) -> Dict[str, Any]:
return get_account_store(data_dir, account_id).load()
def save_account(data_dir: Path, account_id: str, state: Dict[str, Any]) -> None:
get_account_store(data_dir, account_id).save(state)
def account_data_dir(data_dir: Path, account_id: str) -> Path:
"""Per-account directory for channel DBs, media, exports."""
return data_dir / "accounts" / account_id
def account_session_path(session_dir: Path, account_id: str) -> str:
"""Per-account Telethon session file. Always under session/<id>.session."""
return str(session_dir / f"{account_id}.session")
# ── MIGRATION (with data copy) ─────────────────────────────────────────
def _copy_channel_data(src_root: Path, dst_root: Path, channel_id: str) -> None:
"""Copy a single channel's DB + media from src_root to dst_root."""
src_ch = src_root / channel_id
dst_ch = dst_root / channel_id
if not src_ch.exists():
return
dst_ch.mkdir(parents=True, exist_ok=True)
# SQLite DB
db_name = f"{channel_id}.db"
src_db = src_ch / db_name
dst_db = dst_ch / db_name
if src_db.exists() and not dst_db.exists():
shutil.copy2(src_db, dst_db)
logger.info(" copied %s", db_name)
# Media directory
src_media = src_ch / "media"
dst_media = dst_ch / "media"
if src_media.exists() and src_media.is_dir():
dst_media.mkdir(parents=True, exist_ok=True)
for item in src_media.iterdir():
if item.is_file():
dst_file = dst_media / item.name
if not dst_file.exists():
shutil.copy2(item, dst_file)
# Exports (csv, json)
for ext in (".csv", ".json"):
for f in src_ch.glob(f"*{ext}"):
dst_f = dst_ch / f.name
if not dst_f.exists():
shutil.copy2(f, dst_f)
def migrate_legacy_state(data_dir: Path, session_dir: Path) -> bool:
"""
Run once at startup. Detects legacy data/state.json that has api_id
and migrates to multi-account format, COPYING channel DBs/media from
data/<channel_id>/ → data/accounts/default/<channel_id>/ .
Returns True if migration was performed.
"""
state_path = data_dir / "state.json"
if not state_path.exists():
return False
try:
with state_path.open("r", encoding="utf-8") as f:
raw: Dict[str, Any] = json.load(f)
except (json.JSONDecodeError, OSError):
return False
# Already migrated or fresh install with no legacy api_id
if raw.get("version") == 2 or not raw.get("api_id"):
return False
logger.info(
"=== Legacy state detected — migrating to multi-account format ==="
)
logger.info("This may take a while if you have many channels with media.")
# ── 1. Create per-account state for "default" ──────────────────────
acc_dir = data_dir / "accounts" / "default"
acc_dir.mkdir(parents=True, exist_ok=True)
acc_state = {
"label": "Default",
"api_id": raw.get("api_id"),
"api_hash": raw.get("api_hash"),
"channels": raw.get("channels", {}),
"channel_names": raw.get("channel_names", {}),
"scrape_media": raw.get("scrape_media", True),
"forwarding_rules": raw.get("forwarding_rules", []),
"continuous_scraping": raw.get(
"continuous_scraping",
{
"enabled": True,
"interval_minutes": 1,
"channels": [],
"run_all_tracked": True,
},
),
}
acc_state_path = acc_dir / "state.json"
with open(acc_state_path, "w", encoding="utf-8") as f:
json.dump(acc_state, f, ensure_ascii=False, indent=2)
f.write("\n")
logger.info(" ✓ data/accounts/default/state.json created")
# ── 2. Copy channel data ───────────────────────────────────────────
legacy_channels = list(raw.get("channels", {}).keys())
if legacy_channels:
logger.info(
" Copying %d channel(s) from data/<id>/ → data/accounts/default/<id>/ ...",
len(legacy_channels),
)
for i, channel_id in enumerate(legacy_channels, 1):
logger.info(" [%d/%d] channel %s", i, len(legacy_channels), channel_id)
_copy_channel_data(data_dir, acc_dir, channel_id)
logger.info(" ✓ Channel data copied")
else:
logger.info(" (no tracked channels to copy)")
# ── 3. Rewrite global state.json ───────────────────────────────────
global_state = {
"accounts": ["default"],
"version": 2,
}
with open(state_path, "w", encoding="utf-8") as f:
json.dump(global_state, f, ensure_ascii=False, indent=2)
f.write("\n")
logger.info(" ✓ data/state.json rewritten (accounts list only)")
# ── 4. Rename legacy session file ─────────────────────────────────
legacy_session = session_dir / "session.session"
if legacy_session.exists():
new_session = session_dir / "default.session"
if not new_session.exists():
legacy_session.rename(new_session)
logger.info(" ✓ session/session.session → session/default.session")
logger.info("=== Migration complete ===")
return True
health.py
+31 -6
View File
@@ -1,6 +1,6 @@
import sqlite3 import sqlite3
from pathlib import Path from pathlib import Path
from typing import Any, Dict from typing import Any, Dict, List, Optional
from app_state import StateStore from app_state import StateStore
@@ -11,6 +11,7 @@ def health_payload(
state_store: StateStore, state_store: StateStore,
continuous_snapshot: Dict[str, Any], continuous_snapshot: Dict[str, Any],
job_queue_size: int, job_queue_size: int,
account_ids: Optional[List[str]] = None,
) -> Dict[str, Any]: ) -> Dict[str, Any]:
checks = { checks = {
"data_dir": _dir_check(data_dir, writable=True), "data_dir": _dir_check(data_dir, writable=True),
@@ -20,6 +21,22 @@ def health_payload(
"continuous": _continuous_check(continuous_snapshot), "continuous": _continuous_check(continuous_snapshot),
"job_queue": {"ok": True, "size": job_queue_size}, "job_queue": {"ok": True, "size": job_queue_size},
} }
# Per-account health
if account_ids:
account_checks: Dict[str, Any] = {}
for acc_id in account_ids:
acc_dir = data_dir / "accounts" / acc_id
session_file = session_dir / f"{acc_id}.session"
account_checks[acc_id] = {
"data_dir": _dir_check(acc_dir),
"session_file": {
"ok": session_file.exists(),
"path": str(session_file),
},
}
checks["accounts"] = account_checks
ok = all(item.get("ok", False) for item in checks.values()) ok = all(item.get("ok", False) for item in checks.values())
return {"ok": ok, "status": "ok" if ok else "degraded", "checks": checks} return {"ok": ok, "status": "ok" if ok else "degraded", "checks": checks}
@@ -63,11 +80,19 @@ def _sqlite_check() -> Dict[str, Any]:
def _continuous_check(snapshot: Dict[str, Any]) -> Dict[str, Any]: def _continuous_check(snapshot: Dict[str, Any]) -> Dict[str, Any]:
status = snapshot.get("status", {}) running_accounts = snapshot.get("running_accounts")
config = snapshot.get("config", {}) if running_accounts is None:
running_accounts = snapshot
return { return {
"ok": True, "ok": True,
"enabled": bool(config.get("enabled")), "account_count": len(running_accounts),
"running": bool(status.get("running")), "accounts": {
"last_error": status.get("last_error"), aid: {
"running": bool((info.get("status") or info).get("running")),
"enabled": bool((info.get("config") or {}).get("enabled", False)),
"last_error": (info.get("status") or info).get("last_error"),
}
for aid, info in running_accounts.items()
if isinstance(info, dict)
},
} }
main.py
+15
View File
@@ -1,5 +1,6 @@
import logging import logging
import sys import sys
from pathlib import Path
from webui_server import run_server from webui_server import run_server
@@ -11,6 +12,20 @@ def main():
datefmt="%Y-%m-%d %H:%M:%S", datefmt="%Y-%m-%d %H:%M:%S",
stream=sys.stdout, stream=sys.stdout,
) )
logger = logging.getLogger(__name__)
# Run legacy migration before starting the server
from app_state import migrate_legacy_state
data_dir = Path("data")
session_dir = Path("session")
try:
if migrate_legacy_state(data_dir, session_dir):
logger.info("Legacy migration completed successfully.")
except Exception as exc:
logger.error("Migration failed: %s", exc)
run_server() run_server()
scraper_jobs.py
+12 -3
View File
@@ -1,6 +1,6 @@
import asyncio import asyncio
import logging import logging
from typing import Any, Dict, List from typing import Any, Dict, List, Optional
from app_state import StateStore from app_state import StateStore
@@ -25,12 +25,21 @@ class ScraperJobService:
asyncio.run(self._run_async(job_type, payload)) asyncio.run(self._run_async(job_type, payload))
async def _run_async(self, job_type: str, payload: Dict[str, Any]) -> None: async def _run_async(self, job_type: str, payload: Dict[str, Any]) -> None:
# Extract account_id from payload, default to None (legacy)
account_id: Optional[str] = payload.get("account_id")
ScraperClass = self._import_scraper_class() ScraperClass = self._import_scraper_class()
scraper = ScraperClass() scraper = ScraperClass(account_id=account_id)
if account_id:
from app_state import load_account
acc_state = load_account(self.state_store.path.parent, account_id)
# Load per-account state into scraper state
scraper.state = acc_state
initialized = await scraper.initialize_client(interactive=False) initialized = await scraper.initialize_client(interactive=False)
if not initialized: if not initialized:
raise RuntimeError( raise RuntimeError(
"Telegram client is not ready. Check credentials, session, and write access to /app/session." "Telegram client is not ready. Check credentials, session, and write access."
) )
try: try:
if job_type == "scrape_channel": if job_type == "scrape_channel":
telegram_scraper_with_forwarding.py
+36 -6
View File
@@ -21,7 +21,14 @@ from telethon.tl.types import (
) )
from telethon.errors import FloodWaitError, SessionPasswordNeededError from telethon.errors import FloodWaitError, SessionPasswordNeededError
import qrcode import qrcode
from app_state import StateStore from app_state import (
StateStore,
account_data_dir,
account_session_path,
get_account_store,
load_account,
save_account,
)
warnings.filterwarnings( warnings.filterwarnings(
"ignore", message="Using async sessions support is an experimental feature" "ignore", message="Using async sessions support is an experimental feature"
@@ -72,14 +79,35 @@ class ForwardingRule:
enabled: bool = True enabled: bool = True
def _ensure_session_wal(session_path: str) -> None:
db_path = session_path if session_path.endswith(".session") else f"{session_path}.session"
if not Path(db_path).exists():
return
try:
conn = sqlite3.connect(db_path, timeout=1)
conn.execute("PRAGMA journal_mode=WAL;")
conn.execute("PRAGMA synchronous=NORMAL;")
conn.close()
except Exception:
pass
class OptimizedTelegramScraper: class OptimizedTelegramScraper:
def __init__(self): def __init__(self, account_id: Optional[str] = None):
self.DATA_DIR = Path("data") self.account_id = account_id
self.SESSION_DIR = Path("session") self.SESSION_DIR = Path("session")
self.DATA_DIR.mkdir(exist_ok=True)
self.SESSION_DIR.mkdir(exist_ok=True) self.SESSION_DIR.mkdir(exist_ok=True)
if account_id:
self.DATA_DIR = Path("data") / "accounts" / account_id
self.state_store = get_account_store(Path("data"), account_id)
else:
self.DATA_DIR = Path("data")
self.state_store = StateStore(self.DATA_DIR / "state.json")
self.DATA_DIR.mkdir(parents=True, exist_ok=True)
self.STATE_FILE = str(self.DATA_DIR / "state.json") self.STATE_FILE = str(self.DATA_DIR / "state.json")
self.state_store = StateStore(self.DATA_DIR / "state.json")
self.state = self.load_state() self.state = self.load_state()
self.client = None self.client = None
self.continuous_scraping_active = False self.continuous_scraping_active = False
@@ -1480,8 +1508,10 @@ class OptimizedTelegramScraper:
print("Invalid API ID. Must be a number.") print("Invalid API ID. Must be a number.")
return False return False
session = account_session_path(self.SESSION_DIR, self.account_id or "session")
_ensure_session_wal(session)
self.client = TelegramClient( self.client = TelegramClient(
str(self.SESSION_DIR / "session"), session,
self.state["api_id"], self.state["api_id"],
self.state["api_hash"], self.state["api_hash"],
) )
webui/app.js
+536 -236
View File
@@ -1,9 +1,17 @@
/* ── State ───────────────────────────────────────────── */
const state = { const state = {
dashboard: null, accounts: [],
auth: null, activeAccount: null,
continuous: null, dashboards: {},
continuous: {},
channels: {},
jobs: {},
authStates: {},
}; };
/* ── API helper ─────────────────────────────────────── */
async function api(path, options = {}) { async function api(path, options = {}) {
const response = await fetch(path, { const response = await fetch(path, {
headers: { "Content-Type": "application/json" }, headers: { "Content-Type": "application/json" },
@@ -16,6 +24,8 @@ async function api(path, options = {}) {
return data; return data;
} }
/* ── Format helpers ─────────────────────────────────── */
function formatDate(value) { function formatDate(value) {
if (!value) return "-"; if (!value) return "-";
return value.replace("T", " ").replace("+00:00", " UTC"); return value.replace("T", " ").replace("+00:00", " UTC");
@@ -58,47 +68,166 @@ function confirmAction(message) {
return window.confirm(message); return window.confirm(message);
} }
function renderAuth(auth) { function isAccountAuthorized(auth) {
document.getElementById("auth-status").textContent = auth.status; const status = auth?.auth_status || auth || {};
document.getElementById("auth-details").textContent = auth.details || ""; return (
auth?.phase === "authorized" ||
auth?.status === "authorized" ||
status.status === "ready" ||
status.status === "authorized"
);
} }
function toggleHidden(id, hidden) { /* ── Account tab management ─────────────────────────── */
document.getElementById(id).classList.toggle("hidden", hidden);
function renderAccountTabs(accounts) {
const container = document.getElementById("account-tabs");
const template = document.getElementById("account-tab-template");
container.innerHTML = "";
accounts.forEach((acc) => {
const node = template.content.firstElementChild.cloneNode(true);
node.dataset.accountId = acc.id;
node.querySelector(".account-tab-name").textContent = acc.label || acc.id;
const statusEl = node.querySelector(".account-tab-status");
if (acc.auth) {
const authOk = isAccountAuthorized(acc.auth);
statusEl.textContent = authOk ? "●" : "○";
statusEl.className = "account-tab-status " + (authOk ? "ok" : "");
}
if (acc.id === state.activeAccount) {
node.classList.add("active");
}
node.addEventListener("click", () => switchAccount(acc.id));
container.appendChild(node);
});
} }
function renderAuthPanel(authData) { function renderAccountPanel(accountId) {
state.auth = authData; const container = document.getElementById("account-panels");
renderAuth(authData.auth_status); const template = document.getElementById("account-panel-template");
const apiId = authData.saved_credentials?.api_id ?? ""; // Don't duplicate
document.getElementById("api-id-input").value = apiId || ""; if (document.getElementById(`panel-${accountId}`)) return;
document.getElementById("api-hash-input").placeholder = authData.saved_credentials?.api_hash_present
? "API Hash saved"
: "API Hash";
const showQr = Boolean(authData.qr_image); const node = template.content.firstElementChild.cloneNode(true);
toggleHidden("qr-wrap", !showQr); node.id = `panel-${accountId}`;
if (showQr) { node.dataset.accountId = accountId;
document.getElementById("qr-image").src = authData.qr_image;
// Add/remove channel form
const addForm = node.querySelector(".add-channel-form");
addForm.addEventListener("submit", async (event) => {
event.preventDefault();
const channelId = addForm.querySelector(".add-channel-id").value.trim();
const name = addForm.querySelector(".add-channel-name").value.trim();
if (!channelId) return;
await api(`/api/accounts/${accountId}/channels/add`, {
method: "POST",
body: JSON.stringify({ channel_id: channelId, name }),
});
addForm.reset();
await refreshAccount(accountId);
});
// Continuous form
const contForm = node.querySelector(".continuous-form");
contForm.addEventListener("submit", async (event) => {
event.preventDefault();
const enabled = contForm.querySelector(".continuous-enabled").checked;
const intervalMinutes = contForm.querySelector(".continuous-interval").value;
const runAllTracked = contForm.querySelector(".continuous-all").checked;
const channels = Array.from(contForm.querySelectorAll(".continuous-channel-checkbox:checked")).map(
(item) => item.value
);
if (!runAllTracked && channels.length === 0 && enabled) {
if (!confirmAction("Continuous scraping enabled with no selected channels. Save anyway?")) return;
}
await api(`/api/accounts/${accountId}/continuous`, {
method: "POST",
body: JSON.stringify({ enabled, interval_minutes: intervalMinutes, run_all_tracked: runAllTracked, channels }),
});
await refreshAccount(accountId);
});
// "All tracked" checkbox toggles individual checkboxes
const contAll = contForm.querySelector(".continuous-all");
contAll.addEventListener("change", () => {
const checkboxes = contForm.querySelectorAll(".continuous-channel-checkbox");
checkboxes.forEach((cb) => { cb.disabled = contAll.checked; });
});
// Content tabs within account panel
const contentTabs = node.querySelectorAll(".content-tab");
contentTabs.forEach((tab) => {
tab.addEventListener("click", () => {
const targetId = tab.dataset.tabTarget;
contentTabs.forEach((t) => t.classList.toggle("active", t === tab));
node.querySelectorAll(".tab-panel").forEach((p) => {
p.classList.toggle("active", p.dataset.panel === targetId);
});
});
});
// Refresh jobs
node.querySelector(".refresh-jobs-btn").addEventListener("click", () => refreshAccount(accountId));
container.appendChild(node);
}
function switchAccount(accountId) {
if (state.activeAccount === accountId) return;
state.activeAccount = accountId;
localStorage.setItem("activeAccount", accountId);
// Update tabs
document.querySelectorAll(".account-tab").forEach((tab) => {
tab.classList.toggle("active", tab.dataset.accountId === accountId);
});
// Show/hide panels
document.querySelectorAll(".account-panel").forEach((panel) => {
panel.classList.toggle("hidden", panel.dataset.accountId !== accountId);
});
// Ensure panel exists
if (!document.getElementById(`panel-${accountId}`)) {
renderAccountPanel(accountId);
} }
toggleHidden("code-form", authData.phase !== "code_required"); // Update sidebar
toggleHidden("password-form", authData.phase !== "password_required"); updateSidebarAccount();
// Refresh data
refreshAccount(accountId);
// Update settings auth section
updateAuthSection(accountId);
} }
function renderSummary(data) { function updateSidebarAccount() {
document.getElementById("channel-count").textContent = String(data.state.channel_count); const acc = state.accounts.find((a) => a.id === state.activeAccount);
document.getElementById("forwarding-count").textContent = String(data.state.forwarding_rules.length); const nameEl = document.getElementById("active-account-name");
const toggle = document.getElementById("scrape-media-toggle"); const statusEl = document.getElementById("active-account-status");
toggle.checked = Boolean(data.state.scrape_media); if (acc) {
document.getElementById("scrape-media-label").textContent = toggle.checked ? "ON" : "OFF"; nameEl.textContent = acc.label || acc.id;
const authOk = isAccountAuthorized(acc.auth);
statusEl.textContent = authOk ? "Authorized session" : "Needs login";
statusEl.style.color = authOk ? "var(--ok)" : "var(--danger)";
} else {
nameEl.textContent = "None";
statusEl.textContent = "Add an account in Settings";
statusEl.style.color = "var(--dim)";
}
} }
function renderChannels(channels) { /* ── Dashboard rendering ────────────────────────────── */
const tbody = document.getElementById("channels-table");
tbody.innerHTML = ""; function renderChannels(accountId, channels) {
const panel = document.getElementById(`panel-${accountId}`);
if (!panel) return;
const tbody = panel.querySelector(".channels-table");
const template = document.getElementById("channel-row-template"); const template = document.getElementById("channel-row-template");
tbody.innerHTML = "";
channels.forEach((channel) => { channels.forEach((channel) => {
const node = template.content.firstElementChild.cloneNode(true); const node = template.content.firstElementChild.cloneNode(true);
@@ -109,50 +238,55 @@ function renderChannels(channels) {
node.querySelector(".last-date").textContent = channel.last_date || "-"; node.querySelector(".last-date").textContent = channel.last_date || "-";
node.querySelector(".scrape-btn").addEventListener("click", async () => { node.querySelector(".scrape-btn").addEventListener("click", async () => {
await api("/api/jobs/scrape", { await api(`/api/accounts/${accountId}/jobs/scrape`, {
method: "POST", method: "POST",
body: JSON.stringify({ channel_id: channel.channel_id }), body: JSON.stringify({ channel_id: channel.channel_id }),
}); });
await refreshDashboard(); await refreshAccount(accountId);
}); });
node.querySelector(".export-btn").addEventListener("click", async () => { node.querySelector(".export-btn").addEventListener("click", async () => {
await api("/api/jobs/export-channel", { await api(`/api/accounts/${accountId}/jobs/export-channel`, {
method: "POST", method: "POST",
body: JSON.stringify({ channel_id: channel.channel_id }), body: JSON.stringify({ channel_id: channel.channel_id }),
}); });
await refreshDashboard(); await refreshAccount(accountId);
}); });
node.querySelector(".export-view-btn").addEventListener("click", () => { node.querySelector(".export-view-btn").addEventListener("click", () => {
window.location.href = `/viewer?channel=${encodeURIComponent(channel.channel_id)}`; window.location.href = `/viewer?channel=${encodeURIComponent(channel.channel_id)}&account=${encodeURIComponent(accountId)}`;
}); });
node.querySelector(".media-btn").addEventListener("click", async () => { node.querySelector(".media-btn").addEventListener("click", async () => {
await api("/api/jobs/rescrape-media", { await api(`/api/accounts/${accountId}/jobs/rescrape-media`, {
method: "POST", method: "POST",
body: JSON.stringify({ channel_id: channel.channel_id }), body: JSON.stringify({ channel_id: channel.channel_id }),
}); });
await refreshDashboard(); await refreshAccount(accountId);
}); });
node.querySelector(".remove-btn").addEventListener("click", async () => { node.querySelector(".remove-btn").addEventListener("click", async () => {
if (!confirmAction(`Remove ${channel.name} from tracked channels?`)) return; if (!confirmAction(`Remove ${channel.name} from tracked channels?`)) return;
await api("/api/channels/remove", { await api(`/api/accounts/${accountId}/channels/remove`, {
method: "POST", method: "POST",
body: JSON.stringify({ channel_id: channel.channel_id }), body: JSON.stringify({ channel_id: channel.channel_id }),
}); });
await refreshDashboard(); await refreshAccount(accountId);
}); });
tbody.appendChild(node); tbody.appendChild(node);
}); });
// Update channel count stat
panel.querySelector(".channel-count").textContent = String(channels.length);
} }
function renderJobs(jobs) { function renderJobs(accountId, jobs) {
const root = document.getElementById("jobs-list"); const panel = document.getElementById(`panel-${accountId}`);
root.innerHTML = ""; if (!panel) return;
const root = panel.querySelector(".jobs-list");
const template = document.getElementById("job-template"); const template = document.getElementById("job-template");
root.innerHTML = "";
if (!jobs.length) { if (!jobs.length) {
root.innerHTML = '<div class="muted">No jobs yet.</div>'; root.innerHTML = '<div class="muted">No jobs yet.</div>';
@@ -171,18 +305,25 @@ function renderJobs(jobs) {
}); });
} }
function renderContinuousChannelPicker(channels, config) { function renderContinuous(accountId, data) {
const root = document.getElementById("continuous-channel-list"); const panel = document.getElementById(`panel-${accountId}`);
if (!panel) return;
const config = data.config || {};
const status = data.status || {};
const contForm = panel.querySelector(".continuous-form");
contForm.querySelector(".continuous-enabled").checked = Boolean(config.enabled);
contForm.querySelector(".continuous-interval").value = config.interval_minutes || 1;
contForm.querySelector(".continuous-all").checked = Boolean(config.run_all_tracked);
// Channel picker
const chList = panel.querySelector(".continuous-channel-list");
const channels = state.channels[accountId] || [];
const runAll = Boolean(config.run_all_tracked); const runAll = Boolean(config.run_all_tracked);
const selected = new Set(config.channels || []); const selected = new Set(config.channels || []);
root.innerHTML = ""; chList.innerHTML = "";
if (!channels.length) { channels.forEach((ch) => {
root.innerHTML = '<div class="muted">No tracked channels.</div>';
return;
}
channels.forEach((channel) => {
const label = document.createElement("label"); const label = document.createElement("label");
label.className = "checkbox-row"; label.className = "checkbox-row";
label.classList.toggle("disabled", runAll); label.classList.toggle("disabled", runAll);
@@ -190,15 +331,59 @@ function renderContinuousChannelPicker(channels, config) {
const checkbox = document.createElement("input"); const checkbox = document.createElement("input");
checkbox.type = "checkbox"; checkbox.type = "checkbox";
checkbox.className = "continuous-channel-checkbox"; checkbox.className = "continuous-channel-checkbox";
checkbox.value = channel.channel_id; checkbox.value = ch.channel_id;
checkbox.checked = runAll || selected.has(channel.channel_id); checkbox.checked = runAll || selected.has(ch.channel_id);
checkbox.disabled = runAll; checkbox.disabled = runAll;
const text = document.createElement("span"); const text = document.createElement("span");
text.textContent = `${channel.name} (${channel.channel_id})`; text.textContent = `${ch.name} (${ch.channel_id})`;
label.append(checkbox, text); label.append(checkbox, text);
root.appendChild(label); chList.appendChild(label);
});
// Meta
panel.querySelector(".continuous-status").textContent = status.running ? "running" : "stopped";
panel.querySelector(".continuous-last-iteration").textContent = status.last_iteration_at
? `${relativeTime(status.last_iteration_at)} (${displayTime(status.last_iteration_at)})`
: "-";
panel.querySelector(".continuous-last-error").textContent = status.last_error || "-";
// Logs
const logsRoot = panel.querySelector(".continuous-logs");
const entries = (status.log_entries || status.logs || []).map(normalizeLogEntry);
logsRoot.innerHTML = "";
if (!entries.length) {
logsRoot.innerHTML = '<div class="log-line"><span class="log-time">-</span><span class="log-level">INFO</span><span class="log-message">No logs yet.</span></div>';
return;
}
entries.slice().reverse().forEach((entry) => {
const row = document.createElement("div");
row.className = `log-line ${entry.level}`;
const time = entry.timestamp ? displayTime(entry.timestamp).split(", ").pop() : entry.legacyTime || "-";
const absolute = entry.timestamp ? displayTime(entry.timestamp) : entry.legacyTime || "-";
row.title = absolute;
const timeNode = document.createElement("span");
timeNode.className = "log-time";
timeNode.textContent = time;
const ageNode = document.createElement("span");
ageNode.className = "log-age";
ageNode.textContent = entry.timestamp ? relativeTime(entry.timestamp) : "-";
const levelNode = document.createElement("span");
levelNode.className = "log-level";
levelNode.textContent = entry.level.toUpperCase();
const messageNode = document.createElement("span");
messageNode.className = "log-message";
messageNode.textContent = entry.message;
row.append(timeNode, ageNode, levelNode, messageNode);
logsRoot.appendChild(row);
}); });
} }
@@ -230,246 +415,361 @@ function normalizeLogEntry(entry) {
}; };
} }
function renderContinuousLogs(status) { function renderSummary(accountId, data) {
const root = document.getElementById("continuous-logs"); const panel = document.getElementById(`panel-${accountId}`);
const entries = (status.log_entries || status.logs || []).map(normalizeLogEntry); if (!panel) return;
root.innerHTML = ""; const d = data.dashboard || data.state || {};
panel.querySelector(".forwarding-count").textContent = String((d.forwarding_rules || []).length);
const toggle = panel.querySelector(".scrape-media-label");
toggle.textContent = d.scrape_media ? "ON" : "OFF";
}
if (!entries.length) { /* ── Account data loading ───────────────────────────── */
const empty = document.createElement("div");
empty.className = "log-line"; async function refreshAccount(accountId) {
empty.innerHTML = '<span class="log-time">-</span><span class="log-age">-</span><span class="log-level">INFO</span><span class="log-message">No logs yet.</span>'; try {
root.appendChild(empty); const [dashboard, authData, continuousData] = await Promise.all([
api(`/api/accounts/${accountId}`),
api(`/api/accounts/${accountId}/auth`).catch(() => ({})),
api(`/api/accounts/${accountId}/continuous`).catch(() => ({})),
]);
state.dashboards[accountId] = dashboard;
state.continuous[accountId] = continuousData;
state.authStates[accountId] = authData;
state.channels[accountId] = dashboard.channels || [];
renderSummary(accountId, dashboard);
renderChannels(accountId, dashboard.channels || []);
renderJobs(accountId, dashboard.jobs || []);
renderContinuous(accountId, continuousData);
// Update tab status indicator
const tab = document.querySelector(`.account-tab[data-account-id="${accountId}"]`);
if (tab) {
const statusEl = tab.querySelector(".account-tab-status");
const authOk = isAccountAuthorized(authData);
statusEl.textContent = authOk ? "●" : "○";
statusEl.className = "account-tab-status " + (authOk ? "ok" : "");
}
} catch (err) {
console.error(`Failed to refresh account ${accountId}:`, err);
}
}
async function loadAccounts() {
let accounts = [];
try {
const resp = await api("/api/accounts");
accounts = resp.accounts || [];
} catch {
// Legacy fallback — check /api/auth
try {
const legacy = await api("/api/auth");
if (legacy.saved_credentials?.api_id) {
accounts = [{ id: "default", label: "Default", auth: legacy.auth_status || legacy }];
}
} catch {
// No accounts at all
}
}
state.accounts = accounts;
if (accounts.length === 0) {
document.getElementById("no-accounts-msg").classList.remove("hidden");
return; return;
} }
entries.slice().reverse().forEach((entry) => { document.getElementById("no-accounts-msg").classList.add("hidden");
const row = document.createElement("div");
row.className = `log-line ${entry.level}`;
const time = entry.timestamp ? displayTime(entry.timestamp).split(", ").pop() : entry.legacyTime || "-";
const absolute = entry.timestamp ? displayTime(entry.timestamp) : entry.legacyTime || "-";
row.title = absolute;
const timeNode = document.createElement("span"); // Render tabs
timeNode.className = "log-time"; renderAccountTabs(accounts);
timeNode.textContent = time;
const ageNode = document.createElement("span"); // Render panels for each account
ageNode.className = "log-age"; accounts.forEach((acc) => renderAccountPanel(acc.id));
ageNode.textContent = entry.timestamp ? relativeTime(entry.timestamp) : "-";
const levelNode = document.createElement("span"); // Determine active account (persisted across reloads)
levelNode.className = "log-level"; const savedId = localStorage.getItem("activeAccount");
levelNode.textContent = entry.level.toUpperCase(); const activeId = (savedId && accounts.some(a => a.id === savedId)) ? savedId : (state.activeAccount || accounts[0].id);
switchAccount(activeId);
const messageNode = document.createElement("span"); // Update settings
messageNode.className = "log-message"; renderSettingsAccounts();
messageNode.textContent = entry.message;
row.append(timeNode, ageNode, levelNode, messageNode);
root.appendChild(row);
});
} }
function renderContinuous(data) { /* ── Settings: Accounts list ────────────────────────── */
state.continuous = data;
const config = data.config || {};
const status = data.status || {};
document.getElementById("continuous-enabled").checked = Boolean(config.enabled); function renderSettingsAccounts() {
document.getElementById("continuous-interval").value = config.interval_minutes || 1; const container = document.getElementById("accounts-list");
document.getElementById("continuous-all").checked = Boolean(config.run_all_tracked); const template = document.getElementById("account-list-item-template");
renderContinuousChannelPicker(state.dashboard?.channels || [], config); container.innerHTML = "";
document.getElementById("continuous-status").textContent = status.running ? "running" : "stopped"; state.accounts.forEach((acc) => {
document.getElementById("continuous-last-iteration").textContent = status.last_iteration_at const node = template.content.firstElementChild.cloneNode(true);
? `${relativeTime(status.last_iteration_at)} (${displayTime(status.last_iteration_at)})` node.querySelector(".account-list-label").textContent = acc.label || acc.id;
: "-"; node.querySelector(".account-list-id").textContent = acc.id;
document.getElementById("continuous-last-error").textContent = status.last_error || "-";
renderContinuousLogs(status);
}
function setupTabs() { const authStatus = node.querySelector(".account-list-auth-status");
const tabs = document.querySelectorAll(".content-tab"); const authOk = isAccountAuthorized(acc.auth);
const panels = document.querySelectorAll(".tab-panel"); authStatus.textContent = authOk ? "Authorized" : "Needs auth";
authStatus.style.color = authOk ? "var(--ok)" : "var(--danger)";
authStatus.style.fontSize = "0.78rem";
tabs.forEach((tab) => { node.querySelector(".account-select-btn").addEventListener("click", () => {
tab.addEventListener("click", () => { switchAccount(acc.id);
const targetId = tab.dataset.tabTarget; document.getElementById("settings-dialog").close();
tabs.forEach((item) => item.classList.toggle("active", item === tab));
panels.forEach((panel) => panel.classList.toggle("active", panel.id === targetId));
}); });
node.querySelector(".account-remove-btn").addEventListener("click", async () => {
if (!confirmAction(`Remove account "${acc.label || acc.id}"? All its data will be deleted.`)) return;
try {
await api(`/api/accounts/${acc.id}`, { method: "DELETE" });
await loadAccounts();
if (state.activeAccount === acc.id) {
state.activeAccount = null;
}
} catch (err) {
alert(`Failed to remove account: ${err.message}`);
}
});
container.appendChild(node);
}); });
} }
async function refreshDashboard() { /* ── Auth (operates on active account) ───────────────── */
const data = await api("/api/dashboard");
const authData = await api("/api/auth"); function updateAuthSection(accountId) {
const continuousData = await api("/api/continuous"); const label = document.getElementById("auth-account-label");
state.dashboard = data; const acc = state.accounts.find((a) => a.id === accountId);
renderAuthPanel(authData); label.textContent = acc ? (acc.label || acc.id) : "-";
renderSummary(data);
renderChannels(data.channels);
renderJobs(data.jobs);
renderContinuous(continuousData);
} }
async function saveCredentials(accountId) {
const apiId = document.getElementById("api-id-input").value.trim();
const apiHash = document.getElementById("api-hash-input").value.trim();
if (!apiId || !apiHash) return;
await api(`/api/accounts/${accountId}/auth/credentials`, {
method: "POST",
body: JSON.stringify({ api_id: apiId, api_hash: apiHash }),
});
await loadAccounts();
}
async function startQrLogin(accountId) {
const data = await api(`/api/accounts/${accountId}/auth/qr/start`, {
method: "POST",
body: JSON.stringify({}),
});
if (data.qr_image) {
document.getElementById("qr-image").src = data.qr_image;
document.getElementById("qr-wrap").classList.remove("hidden");
}
await loadAccounts();
}
async function requestPhoneCode(accountId) {
const phone = document.getElementById("phone-input").value.trim();
if (!phone) return;
await api(`/api/accounts/${accountId}/auth/phone/request`, {
method: "POST",
body: JSON.stringify({ phone }),
});
await loadAccounts();
}
async function submitPhoneCode(accountId) {
const code = document.getElementById("code-input").value.trim();
if (!code) return;
await api(`/api/accounts/${accountId}/auth/phone/submit`, {
method: "POST",
body: JSON.stringify({ code }),
});
document.getElementById("code-input").value = "";
await loadAccounts();
}
async function submitPassword(accountId) {
const password = document.getElementById("password-input").value.trim();
if (!password) return;
await api(`/api/accounts/${accountId}/auth/password`, {
method: "POST",
body: JSON.stringify({ password }),
});
document.getElementById("password-input").value = "";
await loadAccounts();
}
/* ── Scrape / Export all (for active account) ───────── */
async function scrapeAll() {
if (!state.activeAccount) return;
if (!confirmAction("Queue scraping for all tracked channels?")) return;
await api(`/api/accounts/${state.activeAccount}/jobs/scrape`, {
method: "POST",
body: JSON.stringify({}),
});
await refreshAccount(state.activeAccount);
}
async function exportAll() {
if (!state.activeAccount) return;
if (!confirmAction("Queue export for all tracked channels?")) return;
await api(`/api/accounts/${state.activeAccount}/jobs/export`, {
method: "POST",
body: JSON.stringify({}),
});
await refreshAccount(state.activeAccount);
}
async function refreshDialogs() {
if (!state.activeAccount) return;
await api(`/api/accounts/${state.activeAccount}/jobs/refresh-dialogs`, {
method: "POST",
body: JSON.stringify({}),
});
await refreshAccount(state.activeAccount);
}
async function toggleMedia(value) {
if (!state.activeAccount) return;
await api(`/api/accounts/${state.activeAccount}/settings/media`, {
method: "POST",
body: JSON.stringify({ value }),
});
await refreshAccount(state.activeAccount);
}
/* ── Main ───────────────────────────────────────────── */
async function main() { async function main() {
setupTabs(); // ── Sidebar action buttons ──
document.getElementById("scrape-all-btn").addEventListener("click", async () => {
try { await scrapeAll(); } catch (err) { alert("Scrape error: " + err.message); }
});
document.getElementById("export-all-btn").addEventListener("click", async () => {
try { await exportAll(); } catch (err) { alert("Export error: " + err.message); }
});
document.getElementById("refresh-dialogs-btn").addEventListener("click", async () => {
try { await refreshDialogs(); } catch (err) { alert("Dialog refresh error: " + err.message); }
});
// ── Settings dialog ──
const settingsDialog = document.getElementById("settings-dialog"); const settingsDialog = document.getElementById("settings-dialog");
document.getElementById("open-settings-btn").addEventListener("click", () => { document.getElementById("open-settings-btn").addEventListener("click", () => {
// Update auth section for active account
if (state.activeAccount) {
updateAuthSection(state.activeAccount);
}
renderSettingsAccounts();
settingsDialog.showModal(); settingsDialog.showModal();
}); });
document.getElementById("close-settings-btn").addEventListener("click", () => { document.getElementById("close-settings-btn").addEventListener("click", () => {
settingsDialog.close(); settingsDialog.close();
}); });
document.getElementById("scrape-all-btn").addEventListener("click", async (event) => { // ── Add account form ──
if (!confirmAction("Queue scraping for all tracked channels?")) return; document.getElementById("add-account-form").addEventListener("submit", async (event) => {
setBusy(event.currentTarget, true);
try {
await api("/api/jobs/scrape", { method: "POST", body: JSON.stringify({}) });
await refreshDashboard();
} finally {
setBusy(event.currentTarget, false);
}
});
document.getElementById("export-all-btn").addEventListener("click", async (event) => {
if (!confirmAction("Queue export for all tracked channels?")) return;
setBusy(event.currentTarget, true);
try {
await api("/api/jobs/export", { method: "POST", body: JSON.stringify({}) });
await refreshDashboard();
} finally {
setBusy(event.currentTarget, false);
}
});
document.getElementById("refresh-dialogs-btn").addEventListener("click", async (event) => {
setBusy(event.currentTarget, true);
try {
await api("/api/jobs/refresh-dialogs", { method: "POST", body: JSON.stringify({}) });
await refreshDashboard();
} finally {
setBusy(event.currentTarget, false);
}
});
document.getElementById("refresh-jobs-btn").addEventListener("click", refreshDashboard);
document.getElementById("scrape-media-toggle").addEventListener("change", async (event) => {
const checked = event.currentTarget.checked;
document.getElementById("scrape-media-label").textContent = checked ? "ON" : "OFF";
await api("/api/settings/media", {
method: "POST",
body: JSON.stringify({ value: checked }),
});
await refreshDashboard();
});
document.getElementById("add-channel-form").addEventListener("submit", async (event) => {
event.preventDefault(); event.preventDefault();
const channelId = document.getElementById("add-channel-id").value.trim(); const accountId = document.getElementById("add-account-id").value.trim();
const name = document.getElementById("add-channel-name").value.trim(); const label = document.getElementById("add-account-label").value.trim();
if (!channelId) return; const apiId = document.getElementById("add-account-api-id").value.trim();
await api("/api/channels/add", { const apiHash = document.getElementById("add-account-api-hash").value.trim();
method: "POST", if (!accountId) return;
body: JSON.stringify({ channel_id: channelId, name }),
}); try {
event.currentTarget.reset(); await api("/api/accounts", {
await refreshDashboard(); method: "POST",
body: JSON.stringify({
account_id: accountId,
label: label || accountId,
api_id: apiId ? parseInt(apiId) : undefined,
api_hash: apiHash || undefined,
}),
});
event.target.reset();
await loadAccounts();
if (!state.activeAccount) {
switchAccount(accountId);
}
} catch (err) {
alert("Failed to add account: " + err.message);
}
}); });
// ── Credentials form ──
document.getElementById("credentials-form").addEventListener("submit", async (event) => { document.getElementById("credentials-form").addEventListener("submit", async (event) => {
event.preventDefault(); event.preventDefault();
const apiId = document.getElementById("api-id-input").value.trim(); if (!state.activeAccount) return;
const apiHash = document.getElementById("api-hash-input").value.trim(); try {
await api("/api/auth/credentials", { await saveCredentials(state.activeAccount);
method: "POST", } catch (err) {
body: JSON.stringify({ api_id: apiId, api_hash: apiHash }), alert("Failed to save credentials: " + err.message);
}); }
await refreshDashboard();
}); });
// ── QR login ──
document.getElementById("start-qr-btn").addEventListener("click", async (event) => { document.getElementById("start-qr-btn").addEventListener("click", async (event) => {
setBusy(event.currentTarget, true); setBusy(event.currentTarget, true);
try { try {
await api("/api/auth/qr/start", { if (!state.activeAccount) return;
method: "POST", await startQrLogin(state.activeAccount);
body: JSON.stringify({}), } catch (err) {
}); alert("QR login error: " + err.message);
await refreshDashboard();
} finally { } finally {
setBusy(event.currentTarget, false); setBusy(event.currentTarget, false);
} }
}); });
// ── Phone ──
document.getElementById("phone-form").addEventListener("submit", async (event) => { document.getElementById("phone-form").addEventListener("submit", async (event) => {
event.preventDefault(); event.preventDefault();
const phone = document.getElementById("phone-input").value.trim(); if (!state.activeAccount) return;
await api("/api/auth/phone/request", { try {
method: "POST", await requestPhoneCode(state.activeAccount);
body: JSON.stringify({ phone }), } catch (err) {
}); alert("Phone code request error: " + err.message);
await refreshDashboard(); }
}); });
// ── Code ──
document.getElementById("code-form").addEventListener("submit", async (event) => { document.getElementById("code-form").addEventListener("submit", async (event) => {
event.preventDefault(); event.preventDefault();
const code = document.getElementById("code-input").value.trim(); if (!state.activeAccount) return;
await api("/api/auth/phone/submit", { try {
method: "POST", await submitPhoneCode(state.activeAccount);
body: JSON.stringify({ code }), } catch (err) {
}); alert("Code submit error: " + err.message);
event.currentTarget.reset(); }
await refreshDashboard();
}); });
// ── Password ──
document.getElementById("password-form").addEventListener("submit", async (event) => { document.getElementById("password-form").addEventListener("submit", async (event) => {
event.preventDefault(); event.preventDefault();
const password = document.getElementById("password-input").value.trim(); if (!state.activeAccount) return;
await api("/api/auth/password", { try {
method: "POST", await submitPassword(state.activeAccount);
body: JSON.stringify({ password }), } catch (err) {
}); alert("Password error: " + err.message);
event.currentTarget.reset();
await refreshDashboard();
});
document.getElementById("continuous-form").addEventListener("submit", async (event) => {
event.preventDefault();
const enabled = document.getElementById("continuous-enabled").checked;
const intervalMinutes = document.getElementById("continuous-interval").value;
const runAllTracked = document.getElementById("continuous-all").checked;
const channels = Array.from(document.querySelectorAll(".continuous-channel-checkbox:checked")).map(
(item) => item.value,
);
if (!runAllTracked && channels.length === 0 && enabled) {
if (!confirmAction("Continuous scraping is enabled with no selected channels. Save anyway?")) return;
} }
await api("/api/continuous", {
method: "POST",
body: JSON.stringify({
enabled,
interval_minutes: intervalMinutes,
run_all_tracked: runAllTracked,
channels,
}),
});
await refreshDashboard();
}); });
document.getElementById("continuous-all").addEventListener("change", () => { // ── Media toggle ──
renderContinuousChannelPicker(state.dashboard?.channels || [], { document.getElementById("scrape-media-toggle").addEventListener("change", async (event) => {
...(state.continuous?.config || {}), const checked = event.currentTarget.checked;
run_all_tracked: document.getElementById("continuous-all").checked, try { await toggleMedia(checked); } catch (err) { alert("Media toggle error: " + err.message); }
});
}); });
await refreshDashboard(); // ── Load accounts ──
window.setInterval(refreshDashboard, 5000); await loadAccounts();
// ── Periodic refresh ──
window.setInterval(() => {
if (state.activeAccount) {
refreshAccount(state.activeAccount);
}
}, 5000);
} }
main().catch((error) => { main().catch((error) => {
webui/index.html
+180 -110
View File
@@ -4,7 +4,7 @@
<meta charset="UTF-8" /> <meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Telegram Scraper Control Panel</title> <title>Telegram Scraper Control Panel</title>
<link rel="stylesheet" href="/static/style.css?v=2" /> <link rel="stylesheet" href="/static/style.css?v=4" />
</head> </head>
<body> <body>
<div class="app-shell"> <div class="app-shell">
@@ -22,10 +22,10 @@
<a class="nav-link" href="/health">Health</a> <a class="nav-link" href="/health">Health</a>
</nav> </nav>
<section class="status-card"> <section class="status-card" id="active-account-card">
<div class="section-title">Telegram Status</div> <div class="section-title">Active Account</div>
<div id="auth-status" class="status-badge">Checking...</div> <div id="active-account-name" class="status-badge">Loading...</div>
<p id="auth-details" class="muted small"></p> <p id="active-account-status" class="muted small"></p>
</section> </section>
<section class="status-card"> <section class="status-card">
@@ -38,127 +38,54 @@
</aside> </aside>
<main class="content"> <main class="content">
<div class="content-tabs"> <!-- Account Tabs -->
<button class="content-tab active" data-tab-target="overview-tab" type="button">Overview</button> <div id="account-tabs" class="account-tabs"></div>
<button class="content-tab" data-tab-target="continuous-tab" type="button">Continuous</button>
<!-- Account Dashboard Panels -->
<div id="account-panels">
<!-- Each account's dashboard is rendered here dynamically -->
</div> </div>
<section id="overview-tab" class="tab-panel active"> <!-- Fallback for no accounts -->
<section class="panel-grid"> <section id="no-accounts-msg" class="panel hidden">
<div class="panel stat-panel"> <div class="panel-header">
<div class="section-title">Channels</div> <div>
<div id="channel-count" class="stat-value">0</div> <h2>No Accounts Configured</h2>
<p class="muted">Click "Add Account" in Settings to get started, or use the legacy single-account mode.</p>
</div> </div>
<div class="panel stat-panel"> </div>
<div class="section-title">Media scraping</div>
<div id="scrape-media-label" class="stat-value stat-compact">OFF</div>
</div>
<div class="panel stat-panel">
<div class="section-title">Forwarding rules</div>
<div id="forwarding-count" class="stat-value">0</div>
</div>
</section>
<section class="panel">
<div class="panel-header">
<div>
<h2>Tracked Channels</h2>
<p class="muted">Stored in <code>data/state.json</code>. Message stats come from SQLite.</p>
</div>
<form id="add-channel-form" class="inline-form">
<input id="add-channel-id" name="channel_id" placeholder="ID or @username" required />
<input id="add-channel-name" name="name" placeholder="Display name" />
<button class="button primary" type="submit">Add</button>
</form>
</div>
<div class="table-wrap">
<table>
<thead>
<tr>
<th>Channel</th>
<th>Messages</th>
<th>Media</th>
<th>Last message</th>
<th>Actions</th>
</tr>
</thead>
<tbody id="channels-table"></tbody>
</table>
</div>
</section>
<section class="panel jobs-panel">
<div class="panel-header">
<div>
<h2>Jobs</h2>
<p class="muted">Queued work runs one job at a time to keep the Telegram session stable.</p>
</div>
<button class="button" id="refresh-jobs-btn">Refresh</button>
</div>
<div id="jobs-list" class="jobs-list"></div>
</section>
</section>
<section id="continuous-tab" class="tab-panel">
<section class="panel">
<div class="panel-header">
<div>
<h2>Continuous Scraping</h2>
<p class="muted">Runs on a timer and keeps a compact, highlighted run log.</p>
</div>
</div>
<form id="continuous-form" class="continuous-form">
<label class="toggle-row">
<span>Enabled</span>
<span class="switch">
<input id="continuous-enabled" type="checkbox" checked />
<span class="switch-slider"></span>
</span>
</label>
<label class="stack-form">
<span class="muted small">Interval, minutes</span>
<input id="continuous-interval" type="number" min="1" value="1" />
</label>
<label class="toggle-row">
<span>All tracked channels</span>
<span class="switch">
<input id="continuous-all" type="checkbox" checked />
<span class="switch-slider"></span>
</span>
</label>
<div class="channel-picker">
<div class="muted small">Continuous channel set</div>
<div id="continuous-channel-list" class="checkbox-grid"></div>
</div>
<button class="button primary" type="submit">Save continuous settings</button>
</form>
<div class="continuous-meta">
<div>Status: <span id="continuous-status">-</span></div>
<div>Last run: <span id="continuous-last-iteration">-</span></div>
<div>Last error: <span id="continuous-last-error">-</span></div>
</div>
<div id="continuous-logs" class="log-viewer"></div>
</section>
</section> </section>
</main> </main>
</div> </div>
<!-- ════════ Settings Dialog ════════ -->
<dialog id="settings-dialog" class="settings-dialog"> <dialog id="settings-dialog" class="settings-dialog">
<div class="dialog-shell"> <div class="dialog-shell">
<div class="dialog-header"> <div class="dialog-header">
<div> <div>
<div class="eyebrow">Settings</div> <div class="eyebrow">Settings</div>
<h2>Telegram & Scraping</h2> <h2>Global & Accounts</h2>
</div> </div>
<button class="button button-small" id="close-settings-btn" type="button">Close</button> <button class="button button-small" id="close-settings-btn" type="button">Close</button>
</div> </div>
<!-- ── Accounts Management ── -->
<section class="settings-section"> <section class="settings-section">
<div class="section-title">Telegram Login</div> <div class="section-title">Accounts</div>
<div id="accounts-list" class="accounts-list"></div>
<form id="add-account-form" class="stack-form add-account-form">
<input id="add-account-id" name="account_id" placeholder="Account ID (e.g. work)" required />
<input id="add-account-label" name="label" placeholder="Display name (e.g. Work Account)" />
<input id="add-account-api-id" name="api_id" placeholder="API ID" />
<input id="add-account-api-hash" name="api_hash" placeholder="API Hash" />
<button class="button primary" type="submit">Add Account</button>
</form>
</section>
<!-- ── Account Auth ── (shown per account selected in UI) -->
<section class="settings-section" id="account-auth-section">
<div class="section-title">Account Auth: <span id="auth-account-label">-</span></div>
<form id="credentials-form" class="stack-form"> <form id="credentials-form" class="stack-form">
<input id="api-id-input" name="api_id" placeholder="API ID" /> <input id="api-id-input" name="api_id" placeholder="API ID" />
<input id="api-hash-input" name="api_hash" placeholder="API Hash" /> <input id="api-hash-input" name="api_hash" placeholder="API Hash" />
@@ -190,6 +117,7 @@
</form> </form>
</section> </section>
<!-- ── Scraping ── -->
<section class="settings-section"> <section class="settings-section">
<div class="section-title">Scraping</div> <div class="section-title">Scraping</div>
<label class="toggle-row"> <label class="toggle-row">
@@ -203,6 +131,132 @@
</div> </div>
</dialog> </dialog>
<!-- ════════ Templates ════════ -->
<!-- Account Tab -->
<template id="account-tab-template">
<button class="account-tab" type="button">
<span class="account-tab-name"></span>
<span class="account-tab-status"></span>
</button>
</template>
<!-- Account Dashboard Panel -->
<template id="account-panel-template">
<section class="account-panel">
<!-- Dashboard tabs: Overview, Continuous -->
<div class="content-tabs">
<button class="content-tab active" data-tab-target="overview" type="button">Overview</button>
<button class="content-tab" data-tab-target="continuous" type="button">Continuous</button>
</div>
<!-- Overview Tab -->
<div class="tab-panel active" data-panel="overview">
<section class="panel-grid">
<div class="panel stat-panel">
<div class="section-title">Channels</div>
<div class="stat-value channel-count">0</div>
</div>
<div class="panel stat-panel">
<div class="section-title">Media scraping</div>
<div class="stat-value stat-compact scrape-media-label">OFF</div>
</div>
<div class="panel stat-panel">
<div class="section-title">Forwarding rules</div>
<div class="stat-value forwarding-count">0</div>
</div>
</section>
<section class="panel">
<div class="panel-header">
<div>
<h2>Tracked Channels</h2>
<p class="muted">Per-account channel list.</p>
</div>
<form class="inline-form add-channel-form">
<input class="add-channel-id" name="channel_id" placeholder="ID or @username" required />
<input class="add-channel-name" name="name" placeholder="Display name" />
<button class="button primary" type="submit">Add</button>
</form>
</div>
<div class="table-wrap">
<table>
<thead>
<tr>
<th>Channel</th>
<th>Messages</th>
<th>Media</th>
<th>Last message</th>
<th>Actions</th>
</tr>
</thead>
<tbody class="channels-table"></tbody>
</table>
</div>
</section>
<section class="panel jobs-panel">
<div class="panel-header">
<div>
<h2>Jobs</h2>
<p class="muted">Queued work runs one job at a time.</p>
</div>
<button class="button refresh-jobs-btn">Refresh</button>
</div>
<div class="jobs-list"></div>
</section>
</div>
<!-- Continuous Tab -->
<div class="tab-panel" data-panel="continuous">
<section class="panel">
<div class="panel-header">
<div>
<h2>Continuous Scraping</h2>
<p class="muted">Per-account continuous scrape settings.</p>
</div>
</div>
<form class="continuous-form">
<label class="toggle-row">
<span>Enabled</span>
<span class="switch">
<input type="checkbox" class="continuous-enabled" checked />
<span class="switch-slider"></span>
</span>
</label>
<label class="stack-form">
<span class="muted small">Interval, minutes</span>
<input type="number" class="continuous-interval" min="1" value="1" />
</label>
<label class="toggle-row">
<span>All tracked channels</span>
<span class="switch">
<input type="checkbox" class="continuous-all" checked />
<span class="switch-slider"></span>
</span>
</label>
<div class="channel-picker">
<div class="muted small">Continuous channel set</div>
<div class="checkbox-grid continuous-channel-list"></div>
</div>
<button class="button primary" type="submit">Save continuous settings</button>
</form>
<div class="continuous-meta">
<div>Status: <span class="continuous-status">-</span></div>
<div>Last run: <span class="continuous-last-iteration">-</span></div>
<div>Last error: <span class="continuous-last-error">-</span></div>
</div>
<div class="log-viewer continuous-logs"></div>
</section>
</div>
</section>
</template>
<!-- Channel Row -->
<template id="channel-row-template"> <template id="channel-row-template">
<tr> <tr>
<td> <td>
@@ -224,6 +278,7 @@
</tr> </tr>
</template> </template>
<!-- Job Template -->
<template id="job-template"> <template id="job-template">
<article class="job-card"> <article class="job-card">
<div class="job-head"> <div class="job-head">
@@ -235,6 +290,21 @@
</article> </article>
</template> </template>
<script src="/static/app.js"></script> <!-- Account list item in settings -->
<template id="account-list-item-template">
<div class="account-list-item">
<div class="account-list-info">
<span class="account-list-label"></span>
<span class="muted small account-list-id"></span>
<span class="account-list-auth-status"></span>
</div>
<div class="account-list-actions">
<button class="button button-small account-select-btn">Select</button>
<button class="button button-small danger account-remove-btn">Remove</button>
</div>
</div>
</template>
<script src="/static/app.js?v=4"></script>
</body> </body>
</html> </html>
webui/style.css
+303 -98
View File
@@ -1,17 +1,24 @@
:root { :root {
color-scheme: dark; color-scheme: dark;
--bg-color: #050505; --bg-color: #000000;
--panel: #0c0c0c; --panel: #0b0b0b;
--panel-alt: #080808; --panel-solid: #101010;
--text-color: #e0e0e0; --panel-alt: #050505;
--accent: #ffffff; --text-color: #f2f3f5;
--dim: #6f6f6f; --accent: #1fb9aa;
--line: #2a2a2a; --accent-2: #58a6ff;
--danger: #ff6b6b; --bubble-own: #12342f;
--ok: #7ee787; --bubble-other: #171717;
--warn: #f0c674; --dim: #9aa0a6;
--info: #8ab4f8; --line: #242424;
--font-mono: "Courier New", Courier, monospace; --danger: #ff7373;
--ok: #65e6a4;
--warn: #f6c969;
--info: #8ec7ff;
--radius: 18px;
--shadow: none;
--font-ui: "Segoe UI", "Helvetica Neue", Arial, sans-serif;
--font-mono: "IBM Plex Mono", "SFMono-Regular", Consolas, monospace;
} }
* { * {
@@ -20,11 +27,11 @@
body { body {
margin: 0; margin: 0;
background: var(--bg-color); background: #000;
color: var(--text-color); color: var(--text-color);
font-family: var(--font-mono); font-family: var(--font-ui);
font-size: 16px; font-size: 15px;
line-height: 1.55; line-height: 1.5;
} }
a { a {
@@ -35,8 +42,9 @@ a {
a:hover, a:hover,
button:hover { button:hover {
background: var(--text-color); border-color: #333;
color: var(--bg-color); background: #151515;
color: var(--text-color);
} }
code, code,
@@ -50,19 +58,19 @@ button {
height: 100vh; height: 100vh;
overflow: hidden; overflow: hidden;
display: grid; display: grid;
grid-template-columns: 300px minmax(0, 1fr); grid-template-columns: 308px minmax(0, 1fr);
} }
.sidebar { .sidebar {
background: var(--panel-alt); background: #050505;
border-right: 1px dashed var(--line); border-right: 1px solid var(--line);
padding: 28px 24px; padding: 24px 20px;
} }
.content { .content {
overflow-y: auto; overflow-y: auto;
padding: 28px; padding: 24px;
max-width: 1320px; max-width: 1440px;
width: 100%; width: 100%;
} }
@@ -70,21 +78,16 @@ button {
.dialog-header h2 { .dialog-header h2 {
margin: 0; margin: 0;
color: var(--accent); color: var(--accent);
font-size: 1.45rem; font-size: 1.5rem;
text-transform: uppercase; letter-spacing: -0.04em;
letter-spacing: 0;
} }
.eyebrow, .eyebrow,
.section-title { .section-title {
color: var(--dim); color: var(--dim);
font-size: 0.78rem; font-size: 0.75rem;
text-transform: uppercase; text-transform: uppercase;
} letter-spacing: 0.08em;
.eyebrow::before,
.section-title::before {
content: "./";
} }
.muted { .muted {
@@ -112,8 +115,9 @@ input,
.content-tab, .content-tab,
.viewer-channel-item { .viewer-channel-item {
border: 1px solid var(--line); border: 1px solid var(--line);
background: var(--panel); background: #101010;
color: var(--text-color); color: var(--text-color);
border-radius: 999px;
} }
.nav-link, .nav-link,
@@ -125,21 +129,23 @@ input,
min-height: 42px; min-height: 42px;
padding: 9px 13px; padding: 9px 13px;
cursor: pointer; cursor: pointer;
transition: background-color 0.18s ease, border-color 0.18s ease, transform 0.18s ease;
} }
.nav-link.active, .nav-link.active,
.button.primary, .button.primary,
.content-tab.active { .content-tab.active {
border-color: var(--accent); border-color: #0f6f67;
color: var(--accent); color: #ffffff;
box-shadow: inset 3px 0 0 var(--accent); background: #0f6f67;
box-shadow: none;
} }
.nav-link.active:hover, .nav-link.active:hover,
.button.primary:hover, .button.primary:hover,
.content-tab.active:hover { .content-tab.active:hover {
background: var(--panel); background: #128277;
color: var(--accent); color: #ffffff;
} }
.button.danger { .button.danger {
@@ -164,7 +170,9 @@ input,
.message-card, .message-card,
.settings-dialog { .settings-dialog {
background: var(--panel); background: var(--panel);
border: 1px dashed var(--line); border: 1px solid var(--line);
border-radius: var(--radius);
box-shadow: var(--shadow);
} }
.status-card, .status-card,
@@ -185,10 +193,10 @@ input,
.status-badge { .status-badge {
display: inline-flex; display: inline-flex;
padding: 4px 8px; padding: 6px 10px;
border: 1px solid var(--line); border: 1px solid var(--line);
border-radius: 999px;
color: var(--ok); color: var(--ok);
text-transform: uppercase;
} }
.content-tabs { .content-tabs {
@@ -228,8 +236,9 @@ input,
.stat-value { .stat-value {
margin-top: 8px; margin-top: 8px;
font-size: 2rem; font-size: 2.25rem;
font-weight: 700; font-weight: 700;
letter-spacing: -0.05em;
} }
.stat-compact { .stat-compact {
@@ -250,8 +259,9 @@ input,
input { input {
min-width: 160px; min-width: 160px;
padding: 9px 11px; padding: 10px 13px;
outline: 0; outline: 0;
border-radius: 14px;
} }
input:focus { input:focus {
@@ -286,7 +296,8 @@ input:focus {
position: absolute; position: absolute;
inset: 0; inset: 0;
border: 1px solid var(--line); border: 1px solid var(--line);
background: #101010; background: rgba(255, 255, 255, 0.06);
border-radius: 999px;
} }
.switch-slider::before { .switch-slider::before {
@@ -297,6 +308,7 @@ input:focus {
width: 16px; width: 16px;
height: 16px; height: 16px;
background: var(--dim); background: var(--dim);
border-radius: 50%;
transition: transform 0.16s ease, background-color 0.16s ease; transition: transform 0.16s ease, background-color 0.16s ease;
} }
@@ -323,11 +335,11 @@ td {
padding: 12px 10px; padding: 12px 10px;
text-align: left; text-align: left;
vertical-align: top; vertical-align: top;
border-bottom: 1px dashed var(--line); border-bottom: 1px solid var(--line);
} }
tbody tr:hover { tbody tr:hover {
background: #101010; background: #111;
} }
.action-row { .action-row {
@@ -375,7 +387,9 @@ tbody tr:hover {
gap: 10px; gap: 10px;
min-height: 38px; min-height: 38px;
padding: 8px; padding: 8px;
border: 1px dashed var(--line); border: 1px solid var(--line);
border-radius: 14px;
background: #0f0f0f;
} }
.checkbox-row input { .checkbox-row input {
@@ -391,7 +405,8 @@ tbody tr:hover {
max-height: 420px; max-height: 420px;
overflow: auto; overflow: auto;
border: 1px solid var(--line); border: 1px solid var(--line);
background: #030303; background: rgba(0, 0, 0, 0.2);
border-radius: 16px;
} }
.log-line { .log-line {
@@ -404,7 +419,7 @@ tbody tr:hover {
} }
.log-line:hover { .log-line:hover {
background: #101010; background: #111;
} }
.log-time, .log-time,
@@ -455,13 +470,14 @@ tbody tr:hover {
.settings-section { .settings-section {
margin-top: 18px; margin-top: 18px;
padding-top: 16px; padding-top: 16px;
border-top: 1px dashed var(--line); border-top: 1px solid var(--line);
} }
.qr-wrap { .qr-wrap {
margin-top: 14px; margin-top: 14px;
padding: 12px; padding: 12px;
border: 1px dashed var(--line); border: 1px solid var(--line);
border-radius: 16px;
} }
.qr-wrap img { .qr-wrap img {
@@ -481,18 +497,18 @@ tbody tr:hover {
.viewer-shell { .viewer-shell {
display: grid; display: grid;
grid-template-columns: 300px minmax(0, 1fr); grid-template-columns: 360px minmax(0, 1fr);
height: 100vh; height: 100vh;
overflow: hidden; overflow: hidden;
} }
.viewer-sidebar { .viewer-sidebar {
background: var(--panel-alt); background: #050505;
border-right: 1px dashed var(--line); border-right: 1px solid var(--line);
padding: 28px 24px; padding: 14px 10px;
display: flex; display: flex;
flex-direction: column; flex-direction: column;
gap: 20px; gap: 12px;
overflow-y: auto; overflow-y: auto;
} }
@@ -505,48 +521,104 @@ tbody tr:hover {
align-items: flex-start; align-items: flex-start;
justify-content: space-between; justify-content: space-between;
gap: 16px; gap: 16px;
padding: 10px 6px 8px;
} }
.viewer-sidebar-head h1 { .viewer-sidebar-head h1 {
margin: 0; margin: 0;
color: var(--accent); color: var(--accent);
font-size: 1.45rem; font-size: 1.45rem;
text-transform: uppercase; letter-spacing: -0.04em;
letter-spacing: 0;
} }
.viewer-channel-list { .viewer-channel-list {
display: grid; display: grid;
gap: 10px; gap: 2px;
} }
.viewer-channel-item { .viewer-channel-item {
display: grid; display: grid;
gap: 4px; grid-template-columns: 52px minmax(0, 1fr);
align-items: center;
gap: 10px;
width: 100%; width: 100%;
padding: 12px; min-height: 68px;
padding: 8px 10px;
text-align: left; text-align: left;
cursor: pointer; cursor: pointer;
border: 1px solid var(--line); border: 1px solid transparent;
background: var(--panel); background: transparent;
color: var(--text-color); color: var(--text-color);
border-radius: 12px;
} }
.viewer-channel-item.active { .viewer-channel-item.active {
border-color: var(--accent); border-color: transparent;
box-shadow: inset 3px 0 0 var(--accent); background: #12342f;
box-shadow: none;
}
.viewer-channel-item:hover {
background: #111;
}
.viewer-channel-avatar {
display: inline-flex;
align-items: center;
justify-content: center;
width: 48px;
height: 48px;
border-radius: 50%;
background: #5f6b70;
color: white;
font-weight: 700;
letter-spacing: -0.03em;
}
.viewer-channel-copy {
display: grid;
gap: 3px;
min-width: 0;
}
.viewer-channel-row {
display: grid;
grid-template-columns: minmax(0, 1fr) auto;
align-items: center;
gap: 8px;
min-width: 0;
} }
.viewer-channel-name { .viewer-channel-name {
color: var(--accent); color: var(--text-color);
font-weight: 700; font-weight: 700;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
} }
.viewer-channel-meta { .viewer-channel-time,
.viewer-channel-preview {
color: var(--dim); color: var(--dim);
font-size: 0.82rem; font-size: 0.82rem;
} }
.viewer-channel-preview {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.viewer-channel-count {
min-width: 22px;
padding: 2px 7px;
border-radius: 999px;
background: #12342f;
color: var(--accent);
font-size: 0.76rem;
text-align: center;
}
/* ---------- Main panel ---------- */ /* ---------- Main panel ---------- */
.viewer-main { .viewer-main {
@@ -554,27 +626,31 @@ tbody tr:hover {
flex-direction: column; flex-direction: column;
height: 100vh; height: 100vh;
overflow: hidden; overflow: hidden;
background:
radial-gradient(circle at 20px 20px, rgba(255, 255, 255, 0.025) 1px, transparent 1.5px),
radial-gradient(circle at 58px 54px, rgba(255, 255, 255, 0.018) 1px, transparent 1.5px),
#000;
background-size: auto, 84px 84px, 84px 84px, auto;
} }
.viewer-header { .viewer-header {
flex-shrink: 0; flex-shrink: 0;
padding: 20px 28px; padding: 16px 24px;
border-bottom: 1px dashed var(--line); border-bottom: 1px solid var(--line);
background: var(--panel-alt); background: #050505;
} }
.viewer-header h2 { .viewer-header h2 {
margin: 0; margin: 0;
color: var(--accent); color: var(--accent);
font-size: 1.45rem; font-size: 1.45rem;
text-transform: uppercase; letter-spacing: -0.04em;
letter-spacing: 0;
} }
.messages-list { .messages-list {
flex: 1; flex: 1;
overflow-y: auto; overflow-y: auto;
padding: 12px 28px 20px; padding: 18px 34px 22px;
display: flex; display: flex;
flex-direction: column; flex-direction: column;
} }
@@ -594,9 +670,7 @@ tbody tr:hover {
gap: 12px; gap: 12px;
margin: 12px 0; margin: 12px 0;
font-size: 0.78rem; font-size: 0.78rem;
color: var(--dim); color: #d6e6e8;
text-transform: uppercase;
letter-spacing: 0.03em;
flex-shrink: 0; flex-shrink: 0;
} }
@@ -605,7 +679,14 @@ tbody tr:hover {
content: ""; content: "";
flex: 1; flex: 1;
height: 1px; height: 1px;
background: var(--line); background: #242424;
}
.chat-date-sep {
align-self: center;
padding: 5px 12px;
border-radius: 999px;
background: #111;
} }
.chat-date-sep:empty { .chat-date-sep:empty {
@@ -627,15 +708,16 @@ tbody tr:hover {
/* ---------- Chat bubble ---------- */ /* ---------- Chat bubble ---------- */
.chat-bubble { .chat-bubble {
max-width: 75%; max-width: min(680px, 74%);
padding: 7px 11px; padding: 8px 12px 7px;
background: #1c1c1c; background: var(--bubble-other);
border: 1px solid var(--line); border: 1px solid #242424;
border-radius: 1px; border-radius: 18px 18px 18px 6px;
margin-bottom: 4px; margin-bottom: 6px;
overflow-wrap: anywhere; overflow-wrap: anywhere;
word-break: break-word; word-break: break-word;
position: relative; position: relative;
box-shadow: none;
} }
.chat-bubble::before { .chat-bubble::before {
@@ -647,19 +729,20 @@ tbody tr:hover {
.chat-bubble-wrap:not(.chat-bubble-wrap--own) .chat-bubble::before { .chat-bubble-wrap:not(.chat-bubble-wrap--own) .chat-bubble::before {
left: -12px; left: -12px;
border-right-color: #1c1c1c; border-right-color: var(--bubble-other);
border-left: 0; border-left: 0;
} }
.chat-bubble-wrap--own .chat-bubble::before { .chat-bubble-wrap--own .chat-bubble::before {
right: -12px; right: -12px;
border-left-color: #1e3a5f; border-left-color: var(--bubble-own);
border-right: 0; border-right: 0;
} }
.chat-bubble-wrap--own .chat-bubble { .chat-bubble-wrap--own .chat-bubble {
background: #1e3a5f; background: var(--bubble-own);
border-color: #2b5278; border-color: #1f5b52;
border-radius: 18px 18px 6px 18px;
} }
.chat-bubble--highlight { .chat-bubble--highlight {
@@ -675,7 +758,7 @@ tbody tr:hover {
.chat-sender { .chat-sender {
font-size: 0.82rem; font-size: 0.82rem;
color: var(--info); color: var(--accent);
font-weight: 700; font-weight: 700;
margin-bottom: 2px; margin-bottom: 2px;
} }
@@ -690,8 +773,10 @@ tbody tr:hover {
display: flex; display: flex;
gap: 6px; gap: 6px;
margin-bottom: 4px; margin-bottom: 4px;
padding: 4px 0; padding: 6px 8px;
cursor: pointer; cursor: pointer;
border-radius: 10px;
background: rgba(0, 0, 0, 0.18);
} }
.chat-reply.hidden { .chat-reply.hidden {
@@ -702,7 +787,7 @@ tbody tr:hover {
flex-shrink: 0; flex-shrink: 0;
width: 2px; width: 2px;
background: var(--accent); background: var(--accent);
border-radius: 1px; border-radius: 999px;
opacity: 0.5; opacity: 0.5;
} }
@@ -729,6 +814,7 @@ tbody tr:hover {
.chat-text { .chat-text {
white-space: pre-wrap; white-space: pre-wrap;
line-height: 1.45; line-height: 1.45;
font-size: 0.96rem;
} }
.chat-text:empty { .chat-text:empty {
@@ -749,7 +835,8 @@ tbody tr:hover {
.chat-media video { .chat-media video {
display: block; display: block;
max-width: 100%; max-width: 100%;
border: 1px solid var(--line); border: 1px solid rgba(255, 255, 255, 0.08);
border-radius: 14px;
} }
/* ---------- Footer ---------- */ /* ---------- Footer ---------- */
@@ -783,7 +870,7 @@ tbody tr:hover {
} }
.api-method { .api-method {
border-top: 1px dashed var(--line); border-top: 1px solid var(--line);
padding-top: 12px; padding-top: 12px;
} }
@@ -827,7 +914,7 @@ tbody tr:hover {
.sidebar, .sidebar,
.viewer-sidebar { .viewer-sidebar {
border-right: 0; border-right: 0;
border-bottom: 1px dashed var(--line); border-bottom: 1px solid var(--line);
} }
.panel-grid { .panel-grid {
@@ -835,6 +922,124 @@ tbody tr:hover {
} }
} }
/* ── Account Tabs ─────────────────────────────────────── */
.account-tabs {
display: flex;
flex-wrap: wrap;
gap: 8px;
margin-bottom: 16px;
padding-bottom: 10px;
border-bottom: 1px solid var(--line);
}
.account-tab {
display: inline-flex;
align-items: center;
gap: 8px;
min-height: 38px;
padding: 7px 14px;
border: 1px solid var(--line);
border-radius: 999px;
background: #101010;
color: var(--text-color);
cursor: pointer;
font: inherit;
font-size: 0.88rem;
}
.account-tab.active {
border-color: #0f6f67;
color: #ffffff;
background: #0f6f67;
}
.account-tab .account-tab-status {
font-size: 0.72rem;
opacity: 0.7;
}
.account-tab .account-tab-status.ok {
color: var(--ok);
}
.account-tab .account-tab-status.errored {
color: var(--danger);
}
/* ── Account Panels ──────────────────────────────────── */
.account-panel {
display: block;
}
.account-panel.hidden {
display: none;
}
/* ── Settings: Account List ──────────────────────────── */
.accounts-list {
display: grid;
gap: 8px;
margin-bottom: 12px;
}
.account-list-item {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
padding: 10px 12px;
border: 1px solid var(--line);
border-radius: 16px;
background: #0f0f0f;
}
.account-list-info {
display: grid;
gap: 2px;
min-width: 0;
}
.account-list-label {
font-weight: 700;
}
.account-list-id {
font-size: 0.82rem;
}
.account-list-auth-status {
font-size: 0.78rem;
}
.account-list-actions {
display: flex;
gap: 6px;
flex-shrink: 0;
}
.add-account-form {
display: grid;
gap: 8px;
margin-top: 8px;
padding: 12px;
border: 1px solid var(--line);
border-radius: 16px;
background: #0f0f0f;
}
/* ── Active Account Card ─────────────────────────────── */
#active-account-name {
word-break: break-all;
}
#active-account-status {
margin-top: 6px;
}
/* ---------- Mobile: tablet & phone ---------- */ /* ---------- Mobile: tablet & phone ---------- */
@media (max-width: 768px) { @media (max-width: 768px) {
@@ -844,13 +1049,13 @@ tbody tr:hover {
.viewer-sidebar { .viewer-sidebar {
position: fixed; position: fixed;
left: -300px; left: -360px;
top: 0; top: 0;
bottom: 0; bottom: 0;
width: 280px; width: 340px;
z-index: 100; z-index: 100;
transition: left 0.2s ease; transition: left 0.2s ease;
border-right: 1px dashed var(--line); border-right: 1px solid var(--line);
border-bottom: 0; border-bottom: 0;
} }
@@ -939,7 +1144,7 @@ tbody tr:hover {
} }
.viewer-sidebar { .viewer-sidebar {
width: 260px; width: 300px;
padding: 16px; padding: 16px;
} }
webui/viewer.html
+13 -3
View File
@@ -4,7 +4,7 @@
<meta charset="UTF-8" /> <meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Telegram Scraper Viewer</title> <title>Telegram Scraper Viewer</title>
<link rel="stylesheet" href="/static/style.css?v=2" /> <link rel="stylesheet" href="/static/style.css?v=4" />
</head> </head>
<body> <body>
<div class="viewer-shell"> <div class="viewer-shell">
@@ -13,6 +13,7 @@
<div> <div>
<div class="eyebrow">Export Viewer</div> <div class="eyebrow">Export Viewer</div>
<h1>Messages</h1> <h1>Messages</h1>
<p class="muted small">Account: <select id="viewer-account-select"></select></p>
</div> </div>
<button id="sidebar-toggle" class="button button-small sidebar-toggle">☰</button> <button id="sidebar-toggle" class="button button-small sidebar-toggle">☰</button>
<a class="button button-small" href="/">Dashboard</a> <a class="button button-small" href="/">Dashboard</a>
@@ -36,8 +37,17 @@
<template id="viewer-channel-template"> <template id="viewer-channel-template">
<button class="viewer-channel-item"> <button class="viewer-channel-item">
<span class="viewer-channel-name"></span> <span class="viewer-channel-avatar"></span>
<span class="viewer-channel-meta"></span> <span class="viewer-channel-copy">
<span class="viewer-channel-row">
<span class="viewer-channel-name"></span>
<span class="viewer-channel-time"></span>
</span>
<span class="viewer-channel-row">
<span class="viewer-channel-preview"></span>
<span class="viewer-channel-count"></span>
</span>
</span>
</button> </button>
</template> </template>
webui/viewer.js
+126 -12
View File
@@ -1,6 +1,8 @@
if ("scrollRestoration" in history) history.scrollRestoration = "manual"; if ("scrollRestoration" in history) history.scrollRestoration = "manual";
const viewerState = { const viewerState = {
accountId: null,
accounts: [],
channels: [], channels: [],
channelId: null, channelId: null,
oldestMessageId: null, oldestMessageId: null,
@@ -42,6 +44,24 @@ function formatTime(dateStr) {
return d.toLocaleTimeString("en-US", { hour: "2-digit", minute: "2-digit" }); return d.toLocaleTimeString("en-US", { hour: "2-digit", minute: "2-digit" });
} }
function chatListTime(dateStr) {
if (!dateStr) return "";
const d = new Date(dateStr.replace(" ", "T"));
if (isNaN(d.getTime())) return "";
const now = new Date();
if (d.toDateString() === now.toDateString()) {
return d.toLocaleTimeString("en-US", { hour: "2-digit", minute: "2-digit" });
}
return d.toLocaleDateString("en-US", { month: "short", day: "numeric" });
}
function initials(value) {
const source = String(value || "?").replace(/^@/, "").trim();
const words = source.split(/\s+/).filter(Boolean);
if (words.length > 1) return (words[0][0] + words[1][0]).toUpperCase();
return source.slice(0, 2).toUpperCase();
}
function dateKey(dateStr) { function dateKey(dateStr) {
if (!dateStr) return ""; if (!dateStr) return "";
return dateStr.slice(0, 10); return dateStr.slice(0, 10);
@@ -75,8 +95,14 @@ function renderChannelList() {
if (!template) return; if (!template) return;
viewerState.channels.forEach((channel) => { viewerState.channels.forEach((channel) => {
const node = template.content.firstElementChild.cloneNode(true); const node = template.content.firstElementChild.cloneNode(true);
const preview =
channel.last_message_preview ||
(channel.has_database ? "No text in the last saved message" : "No local database yet");
node.querySelector(".viewer-channel-avatar").textContent = initials(channel.name || channel.channel_id);
node.querySelector(".viewer-channel-name").textContent = channel.name; node.querySelector(".viewer-channel-name").textContent = channel.name;
node.querySelector(".viewer-channel-meta").textContent = `${channel.message_count} messages`; node.querySelector(".viewer-channel-time").textContent = chatListTime(channel.last_date);
node.querySelector(".viewer-channel-preview").textContent = preview;
node.querySelector(".viewer-channel-count").textContent = String(channel.message_count || 0);
if (channel.channel_id === viewerState.channelId) { if (channel.channel_id === viewerState.channelId) {
node.classList.add("active"); node.classList.add("active");
} }
@@ -90,6 +116,102 @@ function renderChannelList() {
}); });
} }
function messageEndpoint(channelId, before = "") {
if (viewerState.accountId) {
return `/api/accounts/${encodeURIComponent(viewerState.accountId)}/channels/${encodeURIComponent(channelId)}/messages?limit=80${before}`;
}
return `/api/channels/${encodeURIComponent(channelId)}/messages?limit=80${before}`;
}
function channelsEndpoint() {
if (viewerState.accountId) {
return `/api/accounts/${encodeURIComponent(viewerState.accountId)}/channels`;
}
return "/api/channels";
}
function renderAccountSelector() {
const sel = document.getElementById("viewer-account-select");
if (!sel) return;
sel.innerHTML = "";
const legacy = viewerState.accounts.length === 0;
if (legacy) {
const opt = document.createElement("option");
opt.value = "";
opt.textContent = "Legacy";
sel.appendChild(opt);
} else {
viewerState.accounts.forEach((acc) => {
const opt = document.createElement("option");
opt.value = acc.id;
opt.textContent = acc.label || acc.id;
if (acc.id === viewerState.accountId) opt.selected = true;
sel.appendChild(opt);
});
}
}
async function switchViewerAccount(accountId) {
viewerState.accountId = accountId || null;
viewerState.accounts.forEach((acc) => {
if (acc.id === accountId) {
const opts = document.getElementById("viewer-account-select")?.options;
if (opts) {
for (let i = 0; i < opts.length; i++) {
opts[i].selected = opts[i].value === accountId;
}
}
}
});
try {
const authData = viewerState.accountId
? await api(`/api/accounts/${encodeURIComponent(viewerState.accountId)}/auth`)
: await api("/api/auth");
viewerState.userId = authData.user_id || null;
} catch (e) {
console.warn("Could not fetch user ID:", e);
}
const channels = await api(channelsEndpoint());
viewerState.channels = channels;
viewerState.channelId = channels[0]?.channel_id || null;
viewerState.oldestMessageId = null;
viewerState.newestMessageId = null;
renderChannelList();
if (viewerState.channelId) {
await loadChannel(viewerState.channelId);
}
}
async function loadViewerAccount(params) {
const requested = params.get("account");
try {
const payload = await api("/api/accounts");
viewerState.accounts = payload.accounts || [];
} catch {
viewerState.accounts = [];
}
viewerState.accountId =
requested ||
viewerState.accounts[0]?.id ||
null;
renderAccountSelector();
const sel = document.getElementById("viewer-account-select");
if (sel) {
sel.addEventListener("change", () => switchViewerAccount(sel.value));
}
try {
const authData = viewerState.accountId
? await api(`/api/accounts/${encodeURIComponent(viewerState.accountId)}/auth`)
: await api("/api/auth");
viewerState.userId = authData.user_id || null;
} catch (e) {
console.warn("Could not fetch user ID:", e);
}
}
function buildMessageNode(message, root) { function buildMessageNode(message, root) {
const template = document.getElementById("message-template"); const template = document.getElementById("message-template");
const wrap = template.content.firstElementChild.cloneNode(true); const wrap = template.content.firstElementChild.cloneNode(true);
@@ -240,9 +362,7 @@ async function loadChannel(channelId, append = false) {
? `&before=${viewerState.oldestMessageId}` ? `&before=${viewerState.oldestMessageId}`
: ""; : "";
try { try {
const payload = await api( const payload = await api(messageEndpoint(channelId, before));
`/api/channels/${encodeURIComponent(channelId)}/messages?limit=80${before}`
);
if (append) { if (append) {
const list = document.getElementById("messages-list"); const list = document.getElementById("messages-list");
const prevScrollHeight = list.scrollHeight; const prevScrollHeight = list.scrollHeight;
@@ -285,15 +405,9 @@ function setupInfiniteScroll() {
} }
async function main() { async function main() {
try {
const authData = await api("/api/auth");
viewerState.userId = authData.user_id || null;
} catch (e) {
console.warn("Could not fetch user ID:", e);
}
viewerState.channels = await api("/api/channels");
const params = new URLSearchParams(window.location.search); const params = new URLSearchParams(window.location.search);
await loadViewerAccount(params);
viewerState.channels = await api(channelsEndpoint());
viewerState.channelId = viewerState.channelId =
params.get("channel") || params.get("channel") ||
viewerState.channels[0]?.channel_id || viewerState.channels[0]?.channel_id ||
webui_server.py
+1216 -283
View File
File diff suppressed because it is too large Load Diff