Sign inSign up

systemseed/hosty

By systemseed

•Updated 2 days ago

Hosty CLI for developers.

Image
Developer tools
0

100K+

systemseed/hosty repository overview

⁠Hosty

Hosting infrastructure & DevOps blueprint 🥳

Read the documentation →⁠

⁠Release notes

⁠[v1.5.2]
  • Builds and deployments make sure that the release of a Wodby image Hosty found has the latest version of its software: the version in the tag of the version line (i.e. PHP 8.4.27 in wodby/php:8.4) is compared with the one in the release (8.4.27-r4). When Wodby changed how it names releases, Hosty kept finding the last release named the old way, and a fix of a vulnerability of PHP didn't get to production. A newer version with tags of its own that Hosty doesn't recognise now stops the command; a newer version that Wodby hasn't released yet only gives a warning, and so does a failure to read the versions from Docker Hub. Setting tag of the container in hosty.yaml skips the check.
  • Redis restarts on a deployment only for a new version of Redis. A new tag of its image recreates the pod, so Redis is down for a few seconds and starts empty without persistence, and Wodby published a release of Redis 7 on 21 days of the last year, though only 2 of them had a new version of Redis. A deployment keeps the image that runs in the environment as long as it has the version of the latest release.
⁠[v1.5.1]
  • The code of applications of PHP and Node.js is read-only in their images: it is owned by root, so the processes of the application can't change it, i.e. code that an attacker gets to run through a vulnerability of a site can't leave a backdoor in it. Only the persistent storages stay writable. Node.js ran as the owner of its code; PHP ran as another user only because of the configuration of the image. The image is built in two stages, and the second one gets only the folder of the application, so files that a build hook puts outside of it are left out. An application that writes into its code at runtime or needs such files turns it off with readOnlyCode: false in hosty.yaml. Python keeps writable code for now, as its packages are installed outside of the folder of the application. Tags of images change, so the first deployment of every environment with this version builds its images again.
  • Network policies isolate projects from each other. Pods of a project accept connections only from the load balancers, on the ports of the web servers (80 and 3000), and from pods of their own namespace, and can't reach the file systems of EFS directly. Any pod of the cluster could reach php-fpm of other projects on port 9000, which runs any code with their environment, database credentials included, and the files of all projects on EFS past their access points. tofu apply turns on network policies in the vpc-cni add-on and adds them to all projects.
  • The option unrestricted_access of projects is removed, and with it the old roles create-read-update-delete and create-read-update: all projects are limited to their roles. Before tofu apply, move every project that still has the option: deploy all its environments with Hosty v1.5.0 or newer and remove the option, as described for v1.5.0.
⁠[v1.5.0]
  • Database accounts of projects on MySQL clusters: <project>_development for development environments and <project>_production for production, created by tofu apply with a job only administrators can reach. Environments switch to them on their next deployment, and their import and cleanup no longer need the master user of the database, which can reach all projects of the cluster. The password of an account doesn't change when an environment is created again. Development environments get databases named <project>_development_<environment>_<application>. Run tofu apply before updating Hosty CLI: on MySQL, the new version needs the accounts. Existing development databases have to be renamed to the new names, and their environments deployed right after that: until then they can't reach their databases. Production keeps its names. PostgreSQL clusters work as before.
  • Projects in Terraform are restricted by default. Before tofu apply, set the new option unrestricted_access = true on all existing projects, and remove it from a project once all its environments are deployed with this version. A restricted project has no master user of the database, and is limited to its roles:
    • Only release managers deploy and push images. People keep logs, exec and copy, and hosty run and hosty delete on development environments. Lead developers also have logs, exec, copy, hosty run and hosty backup:create on production.
    • Pod Security baseline is enforced, which rejects privileged pods and access to nodes, and only administrators can delete production.
    • Ingresses serve only domains of the project, with only the annotations of the charts, and images from ECR come only from its repository. Services can only be of the type ClusterIP, and pods can't use direct volumes such as nfs.
    • Jobs of development environments read only partial backups of databases, and backups of databases can't be deleted from the project. Backups of files can, as syncing them removes files.
  • hosty run no longer fails with cannot unmarshal array into Go struct field ... securityContext: empty objects of the template of the pod were sent to Kubernetes as empty lists. Runs are now deleted with their environment, as their finished pods kept its persistent volume claim for a day and a new environment with the same name couldn't start.
⁠[v1.4.4]
  • Alarms for each instance of the database of a cluster: CPU above 85% for 15 minutes and less than 2 GiB of free memory for 15 minutes. They notify the same channels as the other alarms of the cluster and take tofu apply.
⁠[v1.4.3]
  • New setting backup.partial in hosty.yaml: with false, the scheduled backups of a project create only a full dump of the database. It suits projects without development environments, or whose partial dump is the same as the full one, as they dump and store the database twice. Partial backups stay on by default. With them off, a deployment of a new development environment and the --partial options of the backup commands stop with an error, as the import would restore an old partial backup while there is one, and leave the database empty afterwards.
  • backup.hours in hosty.yaml is optional, so that backup may contain only partial.
  • New option database_backup_retention_days of a project in Terraform, to keep its database backups for less time than those of the other projects, up to 30 or from 120 days. Existing backups older than that are deleted too.
⁠[v1.4.2]
  • Deployments of projects with applications of different types, i.e. Node and PHP, no longer fail with nil pointer evaluating interface {}.image since v1.4.0. The charts defined the template of the pods of runs and cron jobs under the same name, and Helm shares names of templates between the charts of a release, so all applications got the template of one type. Projects with applications of a single type aren't affected.
⁠[v1.4.1]
  • Deployments start a new pod before stopping an old one (maxSurge: 1, maxUnavailable: 0). An old pod was stopped first, so a new version that doesn't start left production on one pod, and every deployment halved the capacity while it ran. Now the old pods keep serving until the new ones are ready, and a failed deployment leaves all of them running, with the broken pod next to them until the next deployment. replicas: 1 still stops the old pod first.
  • Topology spread constraints of production pods count only the pods of the same version, so the extra pod of a deployment may run on the node of an old pod, and no node is added for it while the nodes have room.
⁠[v1.4.0]
  • All overrides of an application in hosty.yaml that match the environment apply, in the order of the list. Each matching override was merged into the configuration without overrides, so only the last one took effect, i.e. a production override with hosts was lost when an override with negate matched production too.
  • The build hook of an application stops at the first command that fails, and the build fails with it (set -e). The hook ran as one script that failed only when its last command did, so i.e. a failed composer install followed by a successful command built and pushed an image without the dependencies, and builds of the same commit reused it. A command that may fail on purpose needs || true after it.
  • backup:create runs the backup of the application given with --application. kubectl took only the last of the two selectors of the command, the one of the component, so it ran the backup of whichever application of the project came first, i.e. of "backend" for --application frontend.
  • The tag of an application image depends on the Dockerfile it is built from, so a change of the build hook in hosty.yaml or of the way Hosty builds images builds a new image. The tag depended only on the code, the environment variables and the base image, so i.e. with appRoot in a subfolder a new build hook deployed the image built with the old one. Images of nginx and apache are built again whenever the image of their backend is, as they copy its code. Tags of all images change, so the first deployment of every environment with this version builds its images again.
  • Docker is logged in to AWS ECR before it builds the image of nginx or apache, which starts from the image of the backend in ECR. It was logged in only to push, after the build, so on a machine without the image of the backend (i.e. in CI, when only the web server had to be built again) the build failed with "no basic auth credentials".
  • tag of a container in hosty.yaml pins its base image to a release of the version line, i.e. tag: r10 for wodby/php:8.4-r10, instead of the latest release. The tag was checked against Docker Hub but never used. A tag of Python includes the flavour of its image, so tag: r7 of Python 3.12 is wodby/python:3.12-dev-r7.
  • Values in environment files of applications (environmentFiles) are taken literally. The files were parsed as dotenv files, which expand variables and run commands like a shell, so values with "$", i.e. hashes of passwords ($2y$10$...) that CI writes from its secrets, were cut, and $NAME gave the variables of the CLI. Lines are NAME=value, optionally after export; blank lines and lines starting with # are skipped, and quotes around a whole value are removed. A file that referred to other variables (${NAME}) or ran commands needs the values written out.
  • Docker gets the password of AWS ECR through the standard input (--password-stdin) instead of the arguments of docker login, where other processes could see it, and a failed login printed it with the command line, i.e. to the logs of CI.
  • The debug output (-vvv) no longer shows the credentials of the MFA session (access key, secret key and session token), which gave access to the account for 12 hours to anyone who saw the output. The error about a user without an MFA device names the user, which it left empty.
  • Values of environment variables are escaped in the Dockerfiles of images, which get them exactly as they are. Pairs of single quotes were dropped from values, i.e. 'self' in a Content Security Policy of nginx became self (a host named "self"), and a value with a single quote (i.e. "Bob's") failed the build. A value with a line break, which a Dockerfile can't hold and whose rest became instructions of the Dockerfile, fails with an error that names the variable. Tags of all images change, so the first deployment of every environment with this version builds its images again.
  • backup:files:upload shows how many files of the backup it would upload and delete, and asks for a confirmation before it changes the backup (--yes skips the question and is required without a terminal, i.e. in CI). It refuses a source folder that doesn't exist or is empty. The upload makes the backup the same as the folder, so a wrong or empty folder deleted the files of the backup of production for good, without a question, and a failed upload was reported as done.
  • backup:files:download shows how many files it would download and delete locally, and asks for a confirmation before it deletes local files (--yes skips the question and is required for that without a terminal). It refuses the root of the project and folders outside it. The download makes the folder the same as the backup, so a wrong folder (i.e. "." or "web") lost all local files that the backup didn't have, code included, and a failed download was reported as done.
  • backup:database:list and backup:database:download fail with the error of AWS when they can't list the backups, i.e. when the MFA session has expired or the access is denied. The error was lost in a shell pipe, so it looked like there were no backups. Names of backups with spaces are shown whole. backup:database:download shows the progress of the download and the error when it fails, which it printed only at the end and without the error.
  • logs fails with the error of kubectl, i.e. for a wrong container or an expired login to the cluster, which was lost in a shell pipe, so the command printed nothing and succeeded. It has no time limit, which was a minute, too short for the logs of all replicas of a busy production. Lines are sorted by time only, so lines with the same time (i.e. of a stack trace) keep their order, and fractions of a second of different lengths compare right. logs --watch works without a terminal (i.e. in CI or with the output to a file), and follows up to 50 pods instead of 5, so it no longer fails on production with more replicas, i.e. during a deployment.
  • A deployment of an environment that has never been deployed successfully, i.e. after the import of the backups on the first deployment timed out, fails with the command that removes the history of its release, so that the next deployment installs it again and imports the backups. Helm upgraded such a release instead of installing it, and the backups are imported only on an installation, so the deployment succeeded with an empty or partly imported database and without the files.
  • A deployment of an environment locked by an interrupted one (i.e. a cancelled job of CI) fails with the command that unlocks it, instead of "another operation (install/upgrade/rollback) is in progress" of Helm.
  • backup:create fails as soon as the backup fails, with the reason, the last lines of its logs and the command to see them. It waited only for the backup to complete, so a failed backup kept it waiting for an hour (i.e. a job of CI), and then it failed with "timed out waiting for the condition". A backup still running after an hour fails the command with the command to follow it.
  • Deployments of production work while Terraform replaces the production node group of the cluster (i.e. for another instance type), when it has two of them for a while: the newest one that is neither being created nor deleted is used. Deployments of production required exactly two node groups in the cluster, so all of them failed meanwhile with "Could not receive list of the cluster's node groups".
  • The build hook of an application keeps the caches of Composer, pip, npm and yarn between builds (BuildKit cache mounts), so i.e. composer install doesn't download all packages again after a change of the code: about 40 seconds less for a Drupal site. The caches aren't part of the image. They are kept where Docker keeps its data between builds, i.e. locally or in CI with the cache of Docker layers; elsewhere builds take as long as before.
  • Requests to Docker Hub (for the tags of base images) wait for 15 seconds at most instead of a minute, and are repeated twice when they fail (a network error, an error of Docker Hub or its rate limit), so a single failure no longer fails a build or a deployment. The error says why the request failed. A command asks Docker Hub for each tag once, it asked twice for each image.
  • Names of environments must start and end with a letter or a digit. A name like "feature-" passed the check and failed the deployment in Helm or on the certificate of its domain. A name that becomes "production" only in lower case (i.e. "Production", from a branch of that name) is refused, as it deployed to production.
  • Commands for an existing environment (i.e. shell and logs) find only the environment with the exact name: the check found "stage" when only "stage2" existed, and the command failed later with an unclear error.
  • A version of a container in hosty.yaml that YAML reads as a number without its trailing zero, i.e. 1.30 without quotes (1.3), fails the check with a hint to quote it. It passed the check, and the build failed with "Docker image tag for wodby/nginx not found". Versions like 8.4 keep working without quotes.
  • The -nt shortcut of exec --no-tty is removed: it never worked, as it's read as -n and -t.
  • The unused cli/project.application.Dockerfile and cli/project.webserver.Dockerfile are removed. Dockerfiles of applications are generated by the CLI.
  • The version of Hosty in helm ls and helm history is just the release, i.e. "1.3.1", for a released CLI. Only a build with unreleased changes has the commit it was built from added, i.e. "1.3.1+66c836a" ("1.3.1+local" for an image built locally), instead of "+unreleased". The commit was added to releases too.
  • Commands check for a new release of Hosty on Docker Hub. A new patch release is announced, and the command runs. A new minor or major release stops the command until Hosty is updated. The check waits 3 seconds at most and is skipped when Docker Hub doesn't respond, for builds between releases, and with HOSTY_SKIP_VERSION_CHECK=1 (i.e. for an urgent fix of production before the update).
  • run (environment:run) runs a command in a pod of its own, like a run of a cron job, on the development nodes, i.e. hosty run production -- drush cim -y. It shows the output of the command and exits with its exit code. The run goes on in the cluster when the command is interrupted (i.e. by Ctrl+C or a lost connection), and the command shows how to follow and stop it. A run is stopped after its time limit, an hour by default (--timeout, in seconds, up to a day), isn't repeated when it fails, and is deleted with its output a day after it finishes. --memory asks for more memory (in MiB), and --detach only starts the run. The pod of a run comes from a new template in the charts (podtemplate.run.yaml), the same pod as the one of cron jobs, so an environment needs a deployment with this version first.
  • The release notes of Hosty are shown on Docker Hub: each release updates the overview of systemseed/hosty with the notes of the latest releases from the changelog.
⁠[v1.3.1]
  • The load balancer counts a Node application healthy when /.healthz answers 200, not only 204. Applications answering 200 (i.e. a page or a route of a framework) were never healthy for the load balancer, which then sent requests to all of them regardless of their state; with the readiness gate their new pods would never be ready. The first deployment with this version updates the health check of the target groups.
⁠[v1.3.0]
  • Each run of a cron job is a pod of its own, from the image of the application, with its environment variables, database, search, Redis and storages, instead of a command executed (kubectl exec) in a running pod of the application. A deployment no longer stops the runs that are going: they finish with the code they started with. Runs don't take the memory of the pods of the application, and a run that runs out of its own memory stops alone. The entrypoint of the image prepares the container (i.e. the settings of PHP from PHP_* variables) before the command; the run hook of a Python application doesn't run, so a command needs to prepare what it needs itself.
  • Runs request 100m of CPU and 256 MiB of memory; memory of a job in hosty.yaml (MiB, from 64) requests more. On production, runs of PHP and Python applications have the memory limit of the application, which a job can't ask more than.
  • Runs go to the development nodes: only there on development environments, and there when they have room on production (the production nodes otherwise), to take the load off the nodes of production applications.
  • timeout of a job is counted from the start of its pod and stops the pod (SIGTERM, SIGKILL 30 seconds later). The lifecycle script hosty-cron-exec is removed.
  • The first deployment with this version replaces the CronJobs of environments. The service accounts cronjob and their permission to execute commands in pods (Terraform) are no longer used; they are removed in a later version, once all environments are deployed with this one.
⁠[v1.2.8]
  • exec, shell and copy of the CLI use the newest pod of the application that is ready and not being deleted. Right after a deployment the pods of the previous release are still running while they stop (up to a minute), so a command after deploy (i.e. drush deploy) could run in one of them, with the previous code, and be stopped with the pod some 20 seconds later. A pod that isn't ready is used only when no pod is ready.
⁠[v1.2.7]
  • Hosty v1.2.6 reported itself as 1.2.5 (in the releases of environments and the versions of their charts), as its changelog listed v1.2.5 first. No other changes: deploy with this version instead of v1.2.6.
⁠[v1.2.6]
  • Application type python: Python without a web server, for applications that serve no requests (i.e. schedulers and scheduled scripts). It has no ingress, domains or nginx, without ingress: false; python:nginx keeps its ingress by default.
  • database.from of an application in hosty.yaml gives it the database of another application of the project (the same environment, the same user, the password from the secret of that application) instead of a database of its own. The import, the backups and the deletion of the database belong to the application that owns it, so the application gets none of them, and the database commands of the CLI point to the owner.
⁠[v1.2.5]
  • Deployments of production applications no longer make the cluster autoscaler add a node. Pods were kept on different nodes by a required pod anti-affinity, which counts the old pod while it is stopping (up to a minute), so the new pod fitted on no node. Now they are kept apart by topology spread constraints (one pod per node, up to the max number of pods), which don't count pods that are stopping: the new pod takes the node of the old one. Pods still prefer different availability zones.
⁠[v1.2.4]
  • replicas of an application in hosty.yaml (1 to 10) runs a fixed number of pods on every environment, without autoscaling. With replicas: 1 a new pod is started only once the previous one has stopped (deployment strategy Recreate), so that two pods never run at the same time, i.e. for a scheduler that may not run twice. Production environments without it are autoscaled as before (2 pods at least).
  • The timeout of a cron job stops its command in the pod of the application, with all the processes the command started (SIGTERM, then SIGKILL 30 seconds later). Before, the timeout stopped only the job, while the command went on in the pod, and the next run could start while it was still running. The job itself gets 2 minutes more than the timeout, for the command to stop.
⁠[v1.2.3]
  • ingress: false of an application in hosty.yaml is for applications that serve no requests, i.e. scheduled scripts: they get neither the ingress (public or internal) nor the service, domains or a web server (nginx or apache, whose image isn't built either), so the load balancers of the cluster don't know them. The database, its import, backups and cron jobs work as before. hosts and the tokens {{ <application>.host.N }} can't be used with it.
  • python.healthCheck.command in hosty.yaml checks the health of the Python container with a command, run by /bin/sh -c in the container, instead of the port of the application: the container is healthy while the command exits with 0 (5 minutes to start, a restart after 3 failures in a row checked every 30 seconds).

Tag summary

Content type

Image

Digest

sha256:9e42b0447…

Size

201.7 MB

Last updated

2 days ago

docker pull systemseed/hosty