postverta/stack repository overview

⁠Postverta Stack

Postverta Stack is a tool to quickly set up reproducible full-stack dev environments. The goal is to make modern applications easy to develop and test. The tool is under active development and there are still quite a few rough edges. Please feel free to submit any bug, feature requests, or general comments to issues.

⁠Features

Similar to tools like docker-compose and vagrant, Stack can easily bootstrap a dev environment with multiple services. There are a few important features that we think make Stack unique:

• No dockerization needed: we realize that it does not make sense to dockerize all services (e.g., lambda functions). Always rebuilding docker images is also a non-trivial overhead for fast iterative development. Therefore, Stack does not require a docker image to be built before running the service. A service can be any code or script written in any language.

• Live code reloading: Stack automatically syncs local code to the containers running the services (we use containers only for the sake of isolation). No need for complex volume mount configurations.

• (Future) online development and testing: currently, all dev services run locally in a VM. In the future, we also plan to support dedicated development cluster for completely online development and testing.

⁠Installation

Stack has been tested under Linux (ubuntu 14.04 and 16.04) and macOS (sierra). Currently, the tool requires minikube⁠ as a local orchestrator of containers.

⁠Install minikube

The following instructions are copied from the official documentation⁠) for your convenience.

⁠Download kubectl

Note: if you are already using kubectl to manage your other kubernetes clusters, the installation of minikube may overwrite your current $HOME/.kube config. Please make a backup of the file before proceeding.

macOS binary:

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl

Linux binary:

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

⁠Make the kubectl executable and move it to your PATH

chmod +x ./kubectl && sudo mv ./kubectl /usr/local/bin/kubectl

⁠Download minikube, a tool to start and stop a local kubernetes cluster

macOS binary:

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.20.0/minikube-darwin-amd64

Linux binary:

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.20.0/minikube-linux-amd64

⁠Make the minikube executable and move it to your PATH

chmod +x minikube && sudo mv minikube /usr/local/bin/minikube

⁠Install the VM driver

macOS (xhyve driver)

$ brew update
$ brew install docker-machine-driver-xhyve

# docker-machine-driver-xhyve need root owner and uid
$ sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
$ sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve

Linux (KVM driver)

# For ubuntu 16.04
$ curl -Lo docker-machine-driver-kvm https://github.com/dhiltgen/docker-machine-kvm/releases/download/v0.10.0/docker-machine-driver-kvm-ubuntu16.04
# For ubuntu 14.04
$ curl -Lo docker-machine-driver-kvm https://github.com/dhiltgen/docker-machine-kvm/releases/download/v0.10.0/docker-machine-driver-kvm-ubuntu14.04

$ chmod +x docker-machine-driver-kvm
$ sudo mv docker-machine-driver-kvm /usr/local/bin/docker-machine-driver-kvm

# Install libvirt and qemu-kvm on your system
$ sudo apt install libvirt-bin qemu-kvm

# Add yourself to the libvirtd group (use libvirt group for rpm based distros) so you don't need to sudo
$ sudo usermod -a -G libvirtd $(whoami)

# Update your current session for the group change to take effect
$ newgrp libvirtd

A special driver called none is needed if you are already running Linux in virtualization (e.g., running a VM in AWS EC2). The driver will not start a virtual machine, but instead will run all containers through the host's own docker engine. There is no dependency of the none driver except for docker⁠

⁠Start minikube.

macOS

minikube start --vm-driver xhyve

Linux (with KVM)

minikube start --vm-driver kvm

Linux (with none driver)

sudo minikube start --vm-driver none
# Because minikube is started with root, need to change the kube config ownership
sudo chown -R $USER $HOME/.kube
sudo chgrp -R $USER $HOME/.kube
sudo chown -R $USER $HOME/.minikube
sudo chgrp -R $USER $HOME/.minikube

If everything works, you should be able to run kubectl get namespace and see something like the following:

NAME          STATUS    AGE
default       Active    6d
kube-public   Active    6d
kube-system   Active    6d

⁠Install Postverta Stack

The current implementation of Stack is only a single python executable file (pv). To install it, simply download the file, make it executable, and copy it to /usr/local/bin:

curl -LO https://raw.githubusercontent.com/postverta/stack/master/pv
chmod +x ./pv
sudo mv ./pv /usr/local/bin/pv

To run pv, you also need the PyYAML library, which can be easily installed with

sudo pip install pyyaml

⁠Example

The example is based on the Sock shop⁠ application, a microservice-based E-commerce website with 11 different services written in different languages and frameworks.

To get started, first clone the overall project:

git clone https://bitbucket.org/postverta/project.git

The project contains a file pv.yaml that describes the stack.

Run pv init in the cloned directory to initialize the local workspace and check-out all source code into the src directory.

$ pv init
Cloning into '/home/angl/project/src/front-end'...
remote: Counting objects: 1003, done.
remote: Compressing objects: 100% (378/378), done.
remote: Total 1003 (delta 554), reused 998 (delta 552)
Receiving objects: 100% (1003/1003), 10.14 MiB | 3.87 MiB/s, done.
Resolving deltas: 100% (554/554), done.
Checking connectivity... done.
Cloning into '/home/angl/project/src/queue-master'...
remote: Counting objects: 243, done.
remote: Compressing objects: 100% (125/125), done.
remote: Total 243 (delta 59), reused 243 (delta 59)
Receiving objects: 100% (243/243), 43.00 KiB | 0 bytes/s, done.
Resolving deltas: 100% (59/59), done.
...

$ ls src
carts  catalogue  front-end  orders  payment  queue-master  shipping  user

Use pv up to bring up the dev stack. For each service described in pv.yaml, the command will sync the corresponding source code in the src directory into a container, and then build and run the service.

$ pv up
Dev environment created
Service front-end started
Service carts-db started
Service carts started
Service shipping started
Service orders started
Service catalogue started
Service user started
Service orders-db started
Service payment started
Service catalogue-db started
Service user-db started

Use pv status to monitor the status of the services.

$ pv status
NAME            STATUS     HOST:PORT (CLUSTER_IP:PORT)
carts           RUNNING    192.168.64.2:31015 (10.0.0.53:80)
carts-db        RUNNING    192.168.64.2:31417 (10.0.0.7:27017)
catalogue       RUNNING    192.168.64.2:30832 (10.0.0.79:80)
catalogue-db    RUNNING    192.168.64.2:30669 (10.0.0.187:3306)
front-end       RUNNING    192.168.64.2:30953 (10.0.0.195:8079)
orders          RUNNING    192.168.64.2:30782 (10.0.0.215:80)
orders-db       RUNNING    192.168.64.2:30861 (10.0.0.169:27017)
payment         RUNNING    192.168.64.2:31861 (10.0.0.247:80)
shipping        RUNNING    192.168.64.2:31578 (10.0.0.204:80)
user            RUNNING    192.168.64.2:32538 (10.0.0.49:80)
user-db         RUNNING    192.168.64.2:31591 (10.0.0.29:27017)

Each service has its own internal IP (CLUSTER_IP), and services can communicate with each other through that IP or host names. Internal IP/port are also mapped to host IP/port for external access. For instance, you can use your browser to open http://192.168.64.2:30953 and see the E-commerce website (some of the services are Java based, and their compilation can take a while).

The running services also stay in sync with the local source code. Any local change will be reflected to the services in real time. For instance, try modifying src/front-end/public/index.html and reloading the E-commerce website to see your changes take effect.

For debugging, use pv log [service name] to check out the running log of any service:

$ pv log front-end
...
...
> [email protected] start /home/dev/front-end
> node server.js

Using local session manager
App now running in development mode on port 8079
Request received: /cart, undefined
Customer ID: Qfs3bOGSOW77bxPtS64OWZNOl8bLavZo
GET /catalogue?size=5 200 528.818 ms - -
GET /catalogue/images/colourful_socks.jpg 200 119.743 ms - 486192

To get a shell into the container running a particular service, use pv ssh [service name]:

$ pv ssh front-end
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

dev@front-end:~$

To shutdown the entire stack, use pv shutdown. This will stop and delete all service containers. It is easy to recreate the stack with pv up again.

⁠Configuration

pv.yaml is the only configuration file. It must be present at the root directory of a workspace. It has a similar structure as docker-compose.yaml. There are two main sections of the file:

• The repos section describes the source repositories. They will be checked-out as part of pv init.

• The services section describes the services in the stack. Each service is based on a repository, and its properties determine how to build and start the service from the source. In the future we might support other type of services, for instance, services directly from pre-built docker images.

⁠Service lifecycle

• Born: the service is started as a container running a base file system image. The file system image is currently non-changeable, and is defined by the Dockerfile in this repository. The base image is ubuntu-based with all the basic development tools installed. In the future we might support using custom docker image as the base file system.

• Initialize: run additional commands to customize the base file system (e.g., apt-get more packages).

• Sync: copy the source code to the container.

• Install dependencies: install any dependency based on the source files like package.json or requirements.txt.

• Build: build the service executables (if needed).

• Running: start the service process(es) inside the container.

⁠Service properties

• repo: (required) specify a repository to be synced over to the container. The name must match with one from the repos section. The files are synced to a directory inside the container specified by the cwd property.

• type: (optional) this chooses from certain templates that change the default values of some properties. For instance, a type mysql would preset build, start, stop, and ports. Of course, you can still override these preset values by explicitly specifying them. The currently supported values are python, javascript, golang, mysql, mongodb. Other values are accepted but have no effect.

• initialize: (optional) a command that is run right after the container is up and running, before source sync. Can be used to install additional system packages. Output can be seen from pv log (same for other commands like start, build).

• cwd: (optional, default $HOME/repo_name) the directory inside the container where the source code is sync'ed to. By default, it is a directory under $HOME named after the repository.

• install-dependencies: (optional) a command that is run after the initial source sync. The language types (e.g., python and javascript) set some reasonable defaults for this property. The working directory of the command would be cwd (same for build, start, and stop commands).

• build: (optional) specify a command to build the service executables.

• start: (required) specify a command to start the service processes.

• stop: (optional, default to kill all processes spawned from start with SIGTERM) specify a command to stop the service process.

• auto-restart: (optional, default true) whether restarts the service after source code change. Set this to false if the service already listens to source code change (e.g., webpack dev server).

• env: (optional) an array of (key, value) environment variables that are set for all commands.

• ports: (optional) an array of port numbers that are exposed to outside. Ports that are only used between services do not need to be listed here. Run pv status to see the port mapping for the exposed ports.

⁠Command reference

• pv init: clone all source repos to local
• pv up: bring up the dev stack
• pv shutdown: turn off the dev stack
• pv status: show the status of the current stack
• pv log [service-name]: get the running log of service-name
• pv ssh [service-name]: access the container of service-name
• pv restart [service-name]: restart the processes of service-name. This is equivalent of running the stop and start commands sequentially.
• pv reset [service-name]: reset the container for service-name. This will restart the entire lifecycle of the service. Useful with a service's configuration has been changed.