jonlatane/jonline repository overview

Jonline is a federated, open source, community-scale social network built with Rust, React and Flutter along with gRPC and HTTP. This image runs Jonline⁠ atop a standard (not slim) Debian server. Jonline will expose its Flutter Web app on ports 80, 8000, and 443 (if TLS is enabled via environment variable), and the Jonline gRPC BE on port 27707.

⁠Run Jonline locally

⁠Makefile-Managed Setup

If you have make, Docker and any Postgres that provides createdb in your PATH, the easiest solution is to use Jonline's Makefiles by cloning its GitHub repo (note the use of environment variables from backend/.env-example⁠ and Make targets from backend/Makefile⁠):

git clone [email protected]:JonLatane/jonline.git
cd jonline

# Run createdb jonline_dev, and run MinIO's Docker image locally in the background
make local_db_create local_minio_create

# Start Jonline's latest image 
docker run --rm --env-file=backend/.env-example \
    -p 27707:27707 -p 80:80 -p 8000:8000 -p 443:443 \
    jonlatane/jonline

To clean up any data when you're done, simply make local_db_delete local_minio_delete.

⁠Postgres Setup

This is left to the reader as Postgres setup is well-documented already and varies per platform (Homebrew, apt, etc.). Ultimately, you'll want to setup a database jonline_dev, user db_user, and password secure_password such that:

• db_user can run DB migrations (you may need to ALTER USER db_user WITH SUPERUSER as your Postgres admin user)
• Postgres will accept connections from db_user from Docker containers

⁠MinIO Setup

The following should setup a local MinIO (S3-compatible blob hosting service) named jonline-dev-minio, backed by a local directory jonline-minio-data in your current working directory:

mkdir jonline-minio-data
docker run -d -p 9000:9000 -p 9090:9090 --name jonline-dev-minio \
   -v $(pwd)/jonline-minio-data:/data \
   -e "MINIO_ROOT_USER=ROOTNAME" \
   -e "MINIO_ROOT_PASSWORD=CHANGEME123" \
   minio/minio server /data --console-address ":9090"

You can access its console at http://localhost:9090⁠.

⁠Run Jonline Locally

The following should spin up the latest Jonline image against your Postgres and MinIO:

docker run --rm -e DATABASE_URL=postgres://db_user:[email protected]/jonline_dev \
   -e MINIO_ENDPOINT=http://host.docker.internal:9000 \
   -e MINIO_REGION= \
   -e MINIO_BUCKET=jonline-dev \
   -e MINIO_ACCESS_KEY=ROOTNAME \
   -e MINIO_SECRET_KEY=CHANGEME123 \
    -p 27707:27707 -p 80:80 -p 8000:8000 -p 443:443 \
    jonlatane/jonline

⁠Environment Variables

Jonline assumes you can "securely" (yeah, okay, K8s secrets aren't "secure" 🤷🏼‍♂️) provide secrets via environment variable.

• DATABASE_URL: The Postgres database URL Jonline should connect to (including credentials).
• MINIO_ENDPOINT, MINIO_REGION, MINIO_BUCKET, MINIO_ACCESS_KEY, MINIO_SECRET_KEY: Amazon S3/MinIO/compatible object store credentials. Jonline Media features are built atop this and MinIO is required for now, though it may eventually be made optional again.
• TLS_KEY, TLS_CERT: TLS key and cert. If not provided, Jonline will not try to start the secure server. If invalid, Jonline will log the errors but still run the other servers on the other ports. If the certs are set, and Tonic is able to configure itself with them, both the port 80 and 8000 web servers will use the HTTP host header to forward any requests from http://header-host[:8000]/path/to to https://header-host/path/to.
  • See generated_certs/README.md⁠ on GitHub for quick HTTPS/TLS setup instructions, either using Cert-Manager (recommended)⁠, some other CA⁠ or your own custom CA⁠. This should be pretty adaptable to any provider that:
    • Offers a K8s service, with LoadBalancer IP assignment and block storage support
    • Has DNS management with APIs that you can use for your domain
    • Has Cert-Manager support for letting the aforementioned K8s configure the aforementioned DNS manager. (Google Cloud, AWS, Azure, DigitalOcean, Scaleway, etc. all fit this bill!)
• CA_CERT (only for advanced users, for custom CAs and/or mutual TLS setups): CA cert for Jonline BE, if you want to set up mutual TLS (as opposed to web-style TLS) on the gRPC service yourself. (Note that the secure web server on port 443 cannot support a custom CA and will probably crash when used in this configuration. The Tonic/gRPC Jonline BE itself should still run, though.)

⁠Server Management via bash

⁠Connecting

⁠Connect via Docker

• Get the Container ID running Jonline from docker container ls.
• Open bash with docker exec -it <ContainerID> bash.

⁠Connect via Kubernetes

• If you use Jonline's Makefile-based K8s deployment system, from your Jonline repo, simply cd deploys and then make deploy_be_shell (or NAMESPACE=my-namespace deploy_be_shell if your namespace isn't the default value of jonline)
  • Otherwise, see deploys/Makefile⁠ for the underlying kubectl command behind the deploy_be_shell target.

⁠Managing Your Deployment

⁠Environment and System Tools

Jonline uses a Debian image as its base, so any Debian-based server admin tools should be easy to install and use.

• To check CPU usage, etc., apt install htop && htop, for instance.
  • The only substantive non-Debian base packages provided in the Jonline server image are curl, psql, and a distribution of grpcurl⁠.
• To see the DATABASE_URL secret: echo $DATABASE_URL.
  • To test your DB and mess around with it, psql $DATABASE_URL.
• To test your own or other Jonline servers with grpcurl, use ./opt/grpcurl (for instance, ./opt/grpcurl jonline.io:27707 list).

⁠Jonline Tools

Jonline tools all live in /opt. They depend on the DATABASE_URL secret and so should work out-of-the-box so long as the server does.

• To assign a user the admin role: ./opt/set_permission my_username admin on
  • This works replacing admin with any of the permissions defined in Jonline's Protocol Documentation⁠. (Case-insensitive)
• To delete expired auth tokens, ./opt/delete_expired_tokens.
  • The standard Jonline K8s deploy does this in a CronJob so you shouldn't need to do it yourself.
• To delete preview images, ./opt/delete_preview_images.
  • The standard Jonline K8s deploy runs a CronJob to generate preview images, in which case they should be re-generated after you delete them.
  • Note that this image cannot generate preview screenshots as-is, so the provided /opt/generate_preview_images binary will not work. Chromium Headless is big, so I made the separate jonline_preview_generator⁠ for that. If you'd rather install Chromium Headless on your main server and use this binary, simply run the commands from deploys/docker/preview_generator/Dockerfile in the Jonline repo⁠.

⁠Ports

Jonline exposes the following ports:

• 27707: The Jonline protocol port. A gRPC (with web and/or TLS) port exposing the Jonline service and gRPC reflection service.
• 443: If TLS_KEY and TLS_CERT are valid, a secure HTTPS server serving either the Tamagui (default) or Flutter UI is spun up on port 443.
• 80 and 8000: Insecure HTTPS servers. If TLS_KEY and TLS_CERT are valid, they redirect to the HTTPS server on port 443. Otherwise, they serve either the Tamagui (default) or Flutter UI.

⁠Included Jonline binaries

• /opt/jonline: The main Jonline server (and container entry point)
• /opt/delete_expired_tokens: Task to delete expired refresh and auth tokens
• /opt/delete_preview_images: One-off task to delete Jonline preview images (usually so jonline_preview_generator⁠ can regenerate them)