jfloff/mcb repository overview

⁠Pluribus microbenchmark

This serves as a microbenchmark for different distributed cross-system consistency models. It's made from a configurable amount of nodes that communicate with each other. The purpose of each node is to send events to other nodes, directly or indirectly (i.e. by relaying events through other nodes). Each server has a single client attached where requests are sent from.

⁠Deployment overview

The system is comprised of 3 entities, namely: Clients, Servers, Sequencer described below.

• Clients: There is at least one client per server. The client is currently an independent instance but multiple clients are implemented as threads in the client instance. Clients interact by pushing or polling updates to one server.
• Servers: Relay events to the edge server connected to the destination client which may notify them immediately or queue events for deliver upon clients request.
• Sequencer: Sequencer of events responsible for assigning global ids.

⁠Deployment scenarios:

We evaluate two scenarios: one, when no information about the system is provided which requires tracking all the dependencies and causality is enforced transparently (Zero knowledge). The second one is obtained when the system administrator provides dependency information which allow to optimize the ordering (User Assisted).

⁠Zero knowledge/Fully transparent

• Benchmark with eventual consistency (no pluribus, only for show what is the maximum performance even if it includes causality violations)
• Benchmark with a global order get using a sequencer (serializable)
• Benchmark with eventual consistency plus pluribus (will produce a serializable output with high/moderate? overhead compared to sequencer)

⁠User Assisted/Semitransparent

• Benchmark with a causal system with a sequencer - (baseline kronus)
• Benchmark with a causal system with a partial order - (baseline saturn/cops)
• Benchmark eventual version plus pluribus plus dependencies - will have better performance compared to kronus and saturn/cops

⁠Executing the benchmark

⁠Requeriments

You need at least a system with Docker and python3 installed for local deployment. Additionally you can provide Google cloud plataform credentials for deploying it at Google cloud engine.

⁠How to use it

Build a docker image called pluribus-mcb:mgmt with the Dockerfile available or, alternativelly, pull the prebuilt images from the Google Cloud Platform private repository with the script pullimages.sh in order to use the mcb.py tool. This tool enables to to generate, deploy, run and gather results from an experiment.

⁠Step 1 - Configure the experiment

Use the mcb.py tool to generate an experiment description file (experiment.yml) with all experiment settings. This file serves as input to the deploy command.

$ docker build --rm -t pluribusmcb-mgmt .
$ docker run --rm -v $(pwd):/code -w /code  pluribusmcb-mgmt python mcb.py gen

Additionally, there are some advanced available options for customizing the experiment:

• exp: Allow passing a file with custom parameters as input. Example --exp experiment.yml. (Default none.)
• duration: duration of the experiment (default: 300)
• max_delay and min_delay: minimum and maximum value of the random delay between requests (default: 0)
• client_delay: delay between requests at the client (default: 0)
• fanout_factor: factor that each requests fanouts from a node, i.e. if its sent simultaneous to few or many nodes. Example: for a fanout_factor = 0 it only sends to one node at a time. (default :0)
• path_length_factor: coverage factor that considers a path to be short of not. Example: for path_length_factor = 0.3 a request is considered short if its received for less or equal than 30% of the system, and long if more than 30%. (default: 0.3)
• long_path_factor: percentage of requests that have long paths (default: 0.5)
• short_path_factor: percentage of requests that have short paths (default: 0.5)
• server_selection: method for selection of servers. At the moment there are two methods:
  • (default) UNIFORM: all servers receive close to the same number of requests
  • HOTSPOTS: there are servers that exponentially receive more requests than others. simulating hotspots.
• nodes: nodes existing in the system. Example with format:[{'id': 'use', 'zone': 'us-east-1'}]

Check code for a more updated list of all parameters

⁠Step 2 - Prepare the deployment environment

**WARNING: Do not start a container in interactive mode and run the commands. Use the format below: 1 docker run = 1 action. **

The command deploy requires a target parameter. There is no default option and the command will fail if a target is not set. The target options are:

• --local: prepare the virtual infrastructure with docker-compose.
• --gce: deploy at the google cloud platform.

Basic usage:

python mcb.py data/localorder_600_3_1_1_push_50_uniform_1_1_30p_100p_0-0_rest_n233651 deploy --gcp

Map the current dir to /code inside the container and set it to be the working dir (-w). That is required because the docker-compose output file will be stored at the working dir.

Option --dev only works locally and mounts the codebase volume into the container.

⁠Step 3 - Run the experiment

The command run requeries a target paramenter. There is no default option and the command will fail if a target is not set.

python mcb.py data/localorder_600_3_1_1_push_50_uniform_1_1_30p_100p_0-0_rest_n233651 run --gcp

⁠Gather

python mcb.py data/localorder_600_3_1_1_push_50_uniform_1_1_30p_100p_0-0_rest_n233651 gather --gcp

⁠Clean

python mcb.py data/localorder_600_3_1_1_push_50_uniform_1_1_30p_100p_0-0_rest_n233651 clean --strong --gcp

We will gather several metrics:

• Throughput
• Visibility Latency

⁠GCP

⁠Setting up an account

1. Create account
2. Create New Project with name pluribus
3. Activate the Container Registry API
4. Go to Cloud Build > Triggers and add triggers:
   • For each of the following tags client, server, sequencer, mgmt:
     • Set name
     • Set trigger type to Branch
     • Set branch name to master
     • Change the Dockerfile path accordingly. Don't forget to add a trailing / at the end.
     • Set the Image name to gcr.io/$PROJECT_ID/github.com/jfloff/pluribus-mcb:<tag-name>
   • Run triggers and check of Cloud Build > History if build got done
5. Go to instances and create a single VM with default options and name debian91-docker. Choose Debian 9 as OS.
   • Enter on the created VM via SSH - directly on GCP panel.
   • First update apt: sudo apt-get update
   • Install docker follow instructions here https://docs.docker.com/install/linux/docker-ce/debian/⁠
   • Install the following packages: sudo apt-get install htop nload rsync iperf3
   • Install the following python packages: pip install docker-py
6. Stop the instance. Go to Images and click on Create Image
   • Name it debian91-docker
   • Use as disk as source, and debian91-docker as the source disk
   • When image finishes creating delete the instance.
7. Replace the project name, search within code for DEFAULT_GCP_PROJECT_ID and use pluribus-XXXXXX. You can get the project id at the top on the project selection dropdown.
8. Go to APIs & Services > Credentials and create a credential for a Service account key:
   • Select the Compute engine default service account
   • Select JSON and create
   • A file should be downloaded automatically to your computer
   • Move it to the repo and rename it pluribus.json
9. Go to IAM & Admin > Quotas and upgrade free trial account (you won't be charged). It should appear as a banner on top of the page - if not, make sure you are on the project original owner, and not someone that was assigned owner.
10. After that, increase the following quotas to whatever you need (e.g. 72):
    • Service Compute Engine API:
      • CPUs
      • CPUs (all regions)
      • In-use IP addresses
      • In-use IP addresses global
    • You should also select the previous metrics for your wanted regions:
      • Global
      • us-central1
    • Here is a template text for the Request description field:

      I am a PhD student that is doing some initial deploys to access the feasibility of using GCP on my research evaluation. For now I am considering a small cluster in a single region, but in the future I will be doing larger deploys.

11. After quota is approved, go to IAM & Admin and add the following users as Owner of the project:

• [email protected]

1. Create VMs in the zones where you intend to have deployments to make sure no errors will occur:

• us-central-1

⁠GSD Cluster

⁠Setting up the base image VM

Create a new VM with the OS, docker and other tools using the Vagrantfile included into the mcb folder. You can customize the VM updating the provision script at the beginning of the file. NOTE: if you generate a new sshkey, dont forget to import it into the VM using ssh-copy-id before creating the image. Export the VM as image to a box file (https://scotch.io/tutorials/how-to-create-a-vagrant-base-box-from-an-existing-one⁠) and place it somewhere all machines can access (NAS or Webserver) Finally, update the gsd.py _gen_vagrantfile method and modify the config.vm.box_url parameter to point to the new location of the updated base box image. To define different virtual machines resource configuraton use the INSTANCE_TYPE_TO_CONFIGURATION map within the gsd.py.

⁠for libvirt use packer

https://github.com/vagrant-libvirt/vagrant-libvirt#create-box⁠ https://aarhusworks.com/2014/08/26/unattended-installation-of-vm-images-with-packer.html⁠ https://github.com/jakobadam/packer-qemu-templates⁠