thevickypedia/ytsync repository overview

Deployments

⁠YTSync

YTSync is a lightweight API, equipped with a Telegram Bot, to download audio/video from a URL or playlist and asynchronously transfer it to a remote server via rsync (ssh).

⁠Features

• Supports both webhooks and long-polling for Telegram
• Users can switch between webhooks and long-polling through the API
• Multi-profile support — isolate API keys, Telegram bot credentials, and storage per profile
• Recurring, schedule-based playlist tracking (hourly/daily/weekly/monthly) via Telegram or API
• Sequential downloads (to prevent requests being throttled/blocked)
• Cold start (delayed) for the first download with a custom cooldown interval
• Built-in retry mechanism with an exponential backoff factor for remote transfers
• Automatically deletes [Optional] local files after transferring to a remote server
• Built-in support to check file presence on the remote server before requesting download/file-transfer
• Queryable download queue and historical checkpoints via API, with configurable checkpoint retention

⁠Environment variables

⁠Server Settings

• host: Hostname to run API server. Defaults to 0.0.0.0 [OR] localhost
• port: Port number to run the API server. Defaults to 4483
• tz: IANA time zone identifier. Defaults to server's local timezone
• log_config: Dict config or filepath for log configuration. Defaults to logging.basicConfig

⁠Profile Settings

• name: Name of the profile.
• apikey: Key to access via API. Required for API access
• bot_chat_id: Telegram chat ID allowed to use this profile's bot. Required for telegram access
• bot_username: Telegram username allowed to use this profile's bot. Required for telegram access

⁠Telegram Settings

• bot_token: Telegram bot token. Required for telegram access
• poll_interval: Number of seconds between each request to poll. Defaults to 2
• bot_webhook: Telegram bot webhook URL. [Optional]
• bot_webhook_ip: Webhook IP address. [Optional]
• bot_endpoint: API endpoint to serve the webhook. Defaults to /telegram-webhook
• bot_secret: Secret key to verify webhook requests. [Optional]
• bot_certificate: Certificate filepath for webhook server (in case of self-signed certificate) [Optional]

⁠yt-dlp Settings [Optional]

• download_tester: Boolean flag to skip download, and create placeholder files for testing. Defaults to False
• cookie_file: Path to the cookie file.
• source_address: IP address for the requesting source.
• proxy_url: URL of the proxy server.

⁠FileIO Settings

• data_dir: Directory to store the database. Defaults to data
• logs_dir: Directory to store logs. Defaults to logs
• audio_dir: Directory to store audio files. Defaults to audio
• video_dir: Directory to store video files. Defaults to video

⁠Concurrency & Tolerance Settings

• max_retries: Maximum number of retries for rsync and telegram polling. Defaults to 10
• max_timeout: Maximum number of seconds to wait before timing out CLI downloads, rsync transfers, and other remote SSH operations. Defaults to 60
• backoff_factor: Back off factor between each retry attempt. Defaults to 3
• max_error_threshold: Percentage of individual URLs to verify before downloading the entire playlist. Defaults to 30
• response_timeout: Maximum number of seconds to wait before timing out the client request. Defaults to 30

⁠Queue Settings

• delayed_start: Perform a cold start (delayed) for the first download. Defaults to False
• next_buffer: Number of seconds to simulate time taken for a download. Defaults to 60
• cooldown_interval: Number of seconds to wait before processing next in queue. Defaults to 300

⁠Analytics Settings [Optional]

• checkpoint_retention: Number of days/weeks/months to retain the checkpoint data (up to 90 days). Defaults to 3d

⁠Remote Settings [Optional]

• remote_host: Hostname [OR] IP address of the remote server.
• remote_user: Username to connect to the remote server.
• remote_path: Directory path on the remote server, to transfer downloaded files to.
• delete_after_sync: Boolean flag to delete local files after transferring to the remote server.

⁠Limitations

• Environment variables can be read from a .env or .json file, but docker context only supports .env files.
• Profiles are created by default within the chosen audio_dir and video_dir directories.
• YTSync does not support concurrent downloads, and will download sequentially to avoid being throttled/blocked by the source server.
• YTSync runs on a single process with a single event loop. Blocking operations (metadata lookups, downloads, database I/O) are offloaded to a thread pool via asyncio.to_thread so the server stays responsive, but downloads themselves are still processed one at a time, in order; transfers are submitted as soon as each file finishes downloading.

⁠SSH setup

ssh-keygen -t ed25519 -N "" -f ~/.ssh/id_ed25519
ssh-copy-id user@receiver_ip
ssh user@receiver_ip

⁠Docker

docker run \
  -v ~/.ssh/id_ed25519:/root/.ssh/id_ed25519:ro \
  -v ~/.ssh/known_hosts:/root/.ssh/known_hosts:ro \
  ytsync

⁠Linting

pre-commit will ensure linting

Requirement

pip install pre-commit

Usage

pre-commit run --all-files

⁠Pypi Package

https://pypi.org/project/YTSync/⁠

⁠License & copyright

© Vignesh Rao

Licensed under the MIT License⁠