docs: update README for multi-account, bump app.js cache version
This commit is contained in:
@@ -3,9 +3,9 @@
|
|||||||
Telegram scraper on top of Telethon with:
|
Telegram scraper on top of Telethon with:
|
||||||
|
|
||||||
- CLI workflow for scraping, exports, media recovery, and forwarding
|
- CLI workflow for scraping, exports, media recovery, and forwarding
|
||||||
- lightweight Web UI served by Python
|
- lightweight multi-account Web UI served by Python
|
||||||
- SQLite storage per tracked chat/channel
|
- SQLite storage per tracked chat/channel
|
||||||
- Docker/Compose setup for "run it and leave it on the server"
|
- Docker/Compose / Kubernetes setup for "run it and leave it on the server"
|
||||||
|
|
||||||
## What It Does
|
## What It Does
|
||||||
|
|
||||||
@@ -15,20 +15,31 @@ Telegram scraper on top of Telethon with:
|
|||||||
- Exports to CSV and JSON
|
- Exports to CSV and JSON
|
||||||
- Supports continuous scraping
|
- Supports continuous scraping
|
||||||
- Supports forwarding rules
|
- Supports forwarding rules
|
||||||
- Lets you browse tracked channels and local exports in a web panel
|
- Multi-account support — multiple Telegram sessions side by side
|
||||||
|
- Lets you browse tracked channels and local exports in a web panel with account tabs
|
||||||
|
|
||||||
## Project Layout
|
## Project Layout
|
||||||
|
|
||||||
```text
|
```text
|
||||||
.
|
.
|
||||||
├── main.py # starts the web server
|
├── main.py # starts the web server
|
||||||
├── webui_server.py # HTTP server + API
|
├── webui_server.py # HTTP server + REST API
|
||||||
├── webui/ # plain HTML/CSS/JS frontend
|
├── webui/ # plain HTML/CSS/JS frontend
|
||||||
|
│ ├── index.html / app.js # main dashboard
|
||||||
|
│ ├── viewer.html / viewer.js # message viewer with account selector
|
||||||
|
│ └── swagger.html / swagger.js # API docs
|
||||||
├── telegram_scraper_with_forwarding.py
|
├── telegram_scraper_with_forwarding.py
|
||||||
|
├── app_state.py # multi-account state management
|
||||||
|
├── scraper_jobs.py # async job runner
|
||||||
|
├── health.py # health check endpoints
|
||||||
├── data/
|
├── data/
|
||||||
│ ├── state.json # shared settings and credentials
|
│ ├── state.json # global state (accounts list)
|
||||||
│ └── <channel_id>/<channel_id>.db # SQLite databases
|
│ └── accounts/<id>/ # per-account data
|
||||||
└── session/ # Telethon session files
|
│ ├── state.json # per-account settings & credentials
|
||||||
|
│ └── <channel_id>/ # SQLite DB + media
|
||||||
|
└── session/
|
||||||
|
├── <account_id>.session # per-account Telethon session
|
||||||
|
└── ... # (WAL journal mode for concurrency)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
@@ -97,11 +108,12 @@ http://<server-ip>:8080
|
|||||||
|
|
||||||
The current web panel includes:
|
The current web panel includes:
|
||||||
|
|
||||||
- tracked channels overview
|
- multi-account support with account tabs
|
||||||
|
- tracked channels overview per account
|
||||||
- background job queue
|
- background job queue
|
||||||
- scrape/export/media actions
|
- scrape/export/media actions per account
|
||||||
- shared `scrape_media` toggle
|
- shared `scrape_media` toggle
|
||||||
- local message viewer powered by SQLite + media files
|
- local message viewer with account selector
|
||||||
- API docs at `/swagger` and `/openapi.json`
|
- API docs at `/swagger` and `/openapi.json`
|
||||||
- health checks at `/health` and `/health/continuous`
|
- health checks at `/health` and `/health/continuous`
|
||||||
|
|
||||||
@@ -113,48 +125,67 @@ If you still want the old interactive mode:
|
|||||||
python telegram_scraper_with_forwarding.py
|
python telegram_scraper_with_forwarding.py
|
||||||
```
|
```
|
||||||
|
|
||||||
## Docker
|
## Docker / Kubernetes
|
||||||
|
|
||||||
Build and run:
|
### Docker Compose
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose up -d --build
|
docker compose up -d --build
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Kubernetes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl apply -f k8s/telegram-scraper.yaml
|
||||||
|
```
|
||||||
|
|
||||||
Then open:
|
Then open:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
http://<server-ip>:8080
|
http://<server-ip>:8080
|
||||||
```
|
```
|
||||||
|
|
||||||
### Volumes
|
### Volumes (Docker)
|
||||||
|
|
||||||
- `./data:/app/data` for databases, exports, and `state.json`
|
- `./data:/app/data` for databases, exports, and `state.json`
|
||||||
- `session:/app/session` for Telethon sessions
|
- `session:/app/session` for Telethon sessions
|
||||||
|
|
||||||
If you already logged in before with the same compose volume and did not remove it, the session should be reused.
|
If you already logged in before with the same compose volume and did not remove it, the session should be reused.
|
||||||
|
|
||||||
|
## Multi-Account
|
||||||
|
|
||||||
|
The app supports multiple Telegram accounts side by side. Each account has:
|
||||||
|
|
||||||
|
- its own `state.json` under `data/accounts/<id>/`
|
||||||
|
- its own Telethon session file `session/<id>.session`
|
||||||
|
- its own channel databases and media
|
||||||
|
|
||||||
|
Use the Web UI account tabs to switch between accounts, or create new ones in Settings.
|
||||||
|
|
||||||
|
Legacy single-account data is automatically migrated to the `default` account on first startup.
|
||||||
|
|
||||||
## Shared State
|
## Shared State
|
||||||
|
|
||||||
The Web UI and CLI share the same files:
|
The Web UI and CLI share the same files:
|
||||||
|
|
||||||
- `data/state.json`
|
- `data/state.json` (global — accounts list)
|
||||||
- `session/session.session`
|
- `data/accounts/<id>/state.json` (per-account credentials and settings)
|
||||||
|
- `session/<id>.session` (per-account Telethon session, WAL journal mode)
|
||||||
|
|
||||||
That means:
|
That means:
|
||||||
|
|
||||||
- credentials can stay in `state.json`
|
- credentials persist in state files
|
||||||
- both entry points use the same tracked channels
|
- both entry points use the same tracked channels per account
|
||||||
- both entry points can reuse the same Telegram login session
|
- both entry points can reuse the same Telegram login sessions
|
||||||
|
|
||||||
## Data Storage
|
## Data Storage
|
||||||
|
|
||||||
### SQLite
|
### SQLite
|
||||||
|
|
||||||
Each tracked channel gets its own database:
|
Each tracked channel per account gets its own database:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
data/<channel_id>/<channel_id>.db
|
data/accounts/<account_id>/<channel_id>/<channel_id>.db
|
||||||
```
|
```
|
||||||
|
|
||||||
Main fields include:
|
Main fields include:
|
||||||
@@ -179,20 +210,22 @@ Main fields include:
|
|||||||
Media files are stored in:
|
Media files are stored in:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
data/<channel_id>/media/
|
data/accounts/<account_id>/<channel_id>/media/
|
||||||
```
|
```
|
||||||
|
|
||||||
### Exports
|
### Exports
|
||||||
|
|
||||||
Generated into the same channel folder:
|
Generated into the same channel folder:
|
||||||
|
|
||||||
- `data/<channel_id>/<channel_id>_<username>.csv`
|
- `data/accounts/<account_id>/<channel_id>/<channel_id>_<username>.csv`
|
||||||
- `data/<channel_id>/<channel_id>_<username>.json`
|
- `data/accounts/<account_id>/<channel_id>/<channel_id>_<username>.json`
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
- The web server is intentionally simple: plain HTML/CSS/JS, no frontend framework
|
- The web server is intentionally simple: plain HTML/CSS/JS, no frontend framework
|
||||||
- The viewer currently reads local SQLite data, not Telegram export HTML directly
|
- The viewer reads local SQLite data and supports account switching via a dropdown
|
||||||
|
- Session SQLite files use WAL journal mode to prevent "database is locked" errors when concurrent scraping and jobs run
|
||||||
|
- The active account tab is persisted in `localStorage` across page reloads
|
||||||
- If dependencies are installed but the Telegram session is missing, the Web UI falls back to read-only status until login is available
|
- If dependencies are installed but the Telegram session is missing, the Web UI falls back to read-only status until login is available
|
||||||
|
|
||||||
## Disclaimer
|
## Disclaimer
|
||||||
|
|||||||
+1
-1
@@ -305,6 +305,6 @@
|
|||||||
</div>
|
</div>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<script src="/static/app.js?v=3"></script>
|
<script src="/static/app.js?v=4"></script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|||||||
Reference in New Issue
Block a user