thevickypedia/ytsync repository overview

Deployments

⁠YTSync

YTSync is a lightweight API, equipped with Telegram Bot to download a playlist and asynchronously transfer it to a remote server via rsync (ssh)

⁠Features

• Supports both webhooks and long-polling
• Users can switch between webhooks and long-polling through API
• Sequential downloads (to prevent requests being throttled/blocked)
• Cold start (delayed) for the first download with a custom cooldown interval
• Built-in retry mechanism with an exponential back off factor for remote transfers
• Automatically deletes [Optional] local files after transferring to a remote server
• Built-in support to check file presence on the remote server before requesting download/file-transfer

⁠Environment variables

⁠Server Settings

• host: Hostname to run API server. Defaults to 0.0.0.0 [OR] localhost
• port: Port number to run the API server. Defaults to 4483
• tz: IANA time zone identifier. Defaults to server's local timezone
• log_config: Dict config or filepath for log configuration. Defaults to logging.basicConfig

⁠Telegram Settings

• bot_token: Telegram bot token. Required for telegram access
• bot_chat_ids: List of bot ids to allow. Required for telegram access
• bot_users: List of bot usernames to allow. Required for telegram access
• poll_interval: Number of seconds between each request to poll. Defaults to 2
• bot_webhook: Telegram bot webhook URL. [Optional]
• bot_webhook_ip: Webhook IP address. [Optional]
• bot_endpoint: API endpoint to serve the webhook. Defaults to /telegram-webhook
• bot_secret: Secret key to verify webhook requests. [Optional]
• bot_certificate: Certificate filepath for webhook server (in case of self-signed certificate) [Optional]

⁠API Settings

• apikey: Key to access via API. Required for API access; defaults to bot_token

⁠yt-dlp Settings [Optional]

• cookie_file: Path to the cookie file.
• source_address: IP address for the requesting source.
• proxy_url: URL of the proxy server.

⁠FileIO Settings

• data_dir: Directory to store the database. Defaults to data
• logs_dir: Directory to store logs. Defaults to logs
• download_dir: Directory to store downloaded files. Defaults to downloads

⁠Concurrency & Tolerance Settings

• max_transfers: Maximum number of concurrent transfers to perform. Defaults to the number of CPU cores
• max_retries: Maximum number of retries for rsync and telegram polling. Defaults to 10
• backoff_factor: Back off factor between each retry attempt. Defaults to 3
• max_error_threshold: Percentage of individual URLs to verify before downloading the entire playlist. Defaults to 30
• response_timeout: Maximum number of seconds to wait before timing out the client request. Defaults to 30

⁠Queue Settings

• delayed_start: Perform a cold start (delayed) for the first download. Defaults to False
• next_buffer: Number of seconds to simulate time taken for a download. Defaults to 60
• cooldown_interval: Number of seconds to wait before processing next in queue. Defaults to 300

⁠Remote Settings [Optional]

• remote_host: Hostname [OR] IP address of the remote server.
• remote_user: Username to connect to the remote server.
• remote_path: Directory path on the remote server, to transfer downloaded files to.
• delete_after_sync: Boolean flag to delete local files after transferring to the remote server.

⁠SSH setup

ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519
ssh-copy-id user@receiver_ip
ssh user@receiver_ip

⁠Docker

docker run \
  -v ~/.ssh/id_ed25519:/root/.ssh/id_ed25519:ro \
  -v ~/.ssh/known_hosts:/root/.ssh/known_hosts:ro \
  ytsync

⁠Linting

pre-commit will ensure linting

Requirement

pip install pre-commit

Usage

pre-commit run --all-files

⁠Pypi Package

https://pypi.org/project/YTSync/⁠

⁠License & copyright

© Vignesh Rao

Licensed under the MIT License⁠