docs: update README for multi-account, bump app.js cache version
This commit is contained in:
@@ -3,9 +3,9 @@
|
||||
Telegram scraper on top of Telethon with:
|
||||
|
||||
- CLI workflow for scraping, exports, media recovery, and forwarding
|
||||
- lightweight Web UI served by Python
|
||||
- lightweight multi-account Web UI served by Python
|
||||
- SQLite storage per tracked chat/channel
|
||||
- Docker/Compose setup for "run it and leave it on the server"
|
||||
- Docker/Compose / Kubernetes setup for "run it and leave it on the server"
|
||||
|
||||
## What It Does
|
||||
|
||||
@@ -15,20 +15,31 @@ Telegram scraper on top of Telethon with:
|
||||
- Exports to CSV and JSON
|
||||
- Supports continuous scraping
|
||||
- Supports forwarding rules
|
||||
- Lets you browse tracked channels and local exports in a web panel
|
||||
- Multi-account support — multiple Telegram sessions side by side
|
||||
- Lets you browse tracked channels and local exports in a web panel with account tabs
|
||||
|
||||
## Project Layout
|
||||
|
||||
```text
|
||||
.
|
||||
├── main.py # starts the web server
|
||||
├── webui_server.py # HTTP server + API
|
||||
├── webui_server.py # HTTP server + REST API
|
||||
├── webui/ # plain HTML/CSS/JS frontend
|
||||
│ ├── index.html / app.js # main dashboard
|
||||
│ ├── viewer.html / viewer.js # message viewer with account selector
|
||||
│ └── swagger.html / swagger.js # API docs
|
||||
├── telegram_scraper_with_forwarding.py
|
||||
├── app_state.py # multi-account state management
|
||||
├── scraper_jobs.py # async job runner
|
||||
├── health.py # health check endpoints
|
||||
├── data/
|
||||
│ ├── state.json # shared settings and credentials
|
||||
│ └── <channel_id>/<channel_id>.db # SQLite databases
|
||||
└── session/ # Telethon session files
|
||||
│ ├── state.json # global state (accounts list)
|
||||
│ └── accounts/<id>/ # per-account data
|
||||
│ ├── state.json # per-account settings & credentials
|
||||
│ └── <channel_id>/ # SQLite DB + media
|
||||
└── session/
|
||||
├── <account_id>.session # per-account Telethon session
|
||||
└── ... # (WAL journal mode for concurrency)
|
||||
```
|
||||
|
||||
## Requirements
|
||||
@@ -97,11 +108,12 @@ http://<server-ip>:8080
|
||||
|
||||
The current web panel includes:
|
||||
|
||||
- tracked channels overview
|
||||
- multi-account support with account tabs
|
||||
- tracked channels overview per account
|
||||
- background job queue
|
||||
- scrape/export/media actions
|
||||
- scrape/export/media actions per account
|
||||
- shared `scrape_media` toggle
|
||||
- local message viewer powered by SQLite + media files
|
||||
- local message viewer with account selector
|
||||
- API docs at `/swagger` and `/openapi.json`
|
||||
- health checks at `/health` and `/health/continuous`
|
||||
|
||||
@@ -113,48 +125,67 @@ If you still want the old interactive mode:
|
||||
python telegram_scraper_with_forwarding.py
|
||||
```
|
||||
|
||||
## Docker
|
||||
## Docker / Kubernetes
|
||||
|
||||
Build and run:
|
||||
### Docker Compose
|
||||
|
||||
```bash
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
### Kubernetes
|
||||
|
||||
```bash
|
||||
kubectl apply -f k8s/telegram-scraper.yaml
|
||||
```
|
||||
|
||||
Then open:
|
||||
|
||||
```text
|
||||
http://<server-ip>:8080
|
||||
```
|
||||
|
||||
### Volumes
|
||||
### Volumes (Docker)
|
||||
|
||||
- `./data:/app/data` for databases, exports, and `state.json`
|
||||
- `session:/app/session` for Telethon sessions
|
||||
|
||||
If you already logged in before with the same compose volume and did not remove it, the session should be reused.
|
||||
|
||||
## Multi-Account
|
||||
|
||||
The app supports multiple Telegram accounts side by side. Each account has:
|
||||
|
||||
- its own `state.json` under `data/accounts/<id>/`
|
||||
- its own Telethon session file `session/<id>.session`
|
||||
- its own channel databases and media
|
||||
|
||||
Use the Web UI account tabs to switch between accounts, or create new ones in Settings.
|
||||
|
||||
Legacy single-account data is automatically migrated to the `default` account on first startup.
|
||||
|
||||
## Shared State
|
||||
|
||||
The Web UI and CLI share the same files:
|
||||
|
||||
- `data/state.json`
|
||||
- `session/session.session`
|
||||
- `data/state.json` (global — accounts list)
|
||||
- `data/accounts/<id>/state.json` (per-account credentials and settings)
|
||||
- `session/<id>.session` (per-account Telethon session, WAL journal mode)
|
||||
|
||||
That means:
|
||||
|
||||
- credentials can stay in `state.json`
|
||||
- both entry points use the same tracked channels
|
||||
- both entry points can reuse the same Telegram login session
|
||||
- credentials persist in state files
|
||||
- both entry points use the same tracked channels per account
|
||||
- both entry points can reuse the same Telegram login sessions
|
||||
|
||||
## Data Storage
|
||||
|
||||
### SQLite
|
||||
|
||||
Each tracked channel gets its own database:
|
||||
Each tracked channel per account gets its own database:
|
||||
|
||||
```text
|
||||
data/<channel_id>/<channel_id>.db
|
||||
data/accounts/<account_id>/<channel_id>/<channel_id>.db
|
||||
```
|
||||
|
||||
Main fields include:
|
||||
@@ -179,20 +210,22 @@ Main fields include:
|
||||
Media files are stored in:
|
||||
|
||||
```text
|
||||
data/<channel_id>/media/
|
||||
data/accounts/<account_id>/<channel_id>/media/
|
||||
```
|
||||
|
||||
### Exports
|
||||
|
||||
Generated into the same channel folder:
|
||||
|
||||
- `data/<channel_id>/<channel_id>_<username>.csv`
|
||||
- `data/<channel_id>/<channel_id>_<username>.json`
|
||||
- `data/accounts/<account_id>/<channel_id>/<channel_id>_<username>.csv`
|
||||
- `data/accounts/<account_id>/<channel_id>/<channel_id>_<username>.json`
|
||||
|
||||
## Notes
|
||||
|
||||
- The web server is intentionally simple: plain HTML/CSS/JS, no frontend framework
|
||||
- The viewer currently reads local SQLite data, not Telegram export HTML directly
|
||||
- The viewer reads local SQLite data and supports account switching via a dropdown
|
||||
- Session SQLite files use WAL journal mode to prevent "database is locked" errors when concurrent scraping and jobs run
|
||||
- The active account tab is persisted in `localStorage` across page reloads
|
||||
- If dependencies are installed but the Telegram session is missing, the Web UI falls back to read-only status until login is available
|
||||
|
||||
## Disclaimer
|
||||
|
||||
+1
-1
@@ -305,6 +305,6 @@
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<script src="/static/app.js?v=3"></script>
|
||||
<script src="/static/app.js?v=4"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Reference in New Issue
Block a user