atosclass/rotterdam-caas

By atosclass

•Updated about 5 years ago

Image

2.1K

Overview Tags

atosclass/rotterdam-caas repository overview

⁠ROTTERDAM CAAS

⁠Description

Rotterdam is a native-cloud Infrastructure-as-a-Service (IaaS) facade which facilitates the deployment and life cycle management of containerized tasks on container orchestration platforms. Its main purpose is to upload, organize, run, manage and stop sets of containers (named tasks) through API calls, and abstract all the resource infrastructure details, even the concept of cluster of machines/instances, to micro-service developers (in the case of CLASS⁠, data analytics application/service developers.

Rotterdam is part of the CLASS Project (Edge and Cloud Computation: A Highly Distributed Software for Big Data Analytics): https://class-project.eu/⁠

⁠Installation Guide

⁠Requirements

Docker (https://docs.docker.com/install/⁠)
Kubernetes (https://kubernetes.io/docs/setup/⁠) / MicroK8s (microKubernetes) (https://microk8s.io/⁠) / Openshift-OKD (https://docs.okd.io/latest/install/index.html⁠)

⁠Installation

You can run Rotterdam in Docker or any other container orchestrator like Kubernetes, by pulling the image from Docker Hub:
```
docker pull atosclass/rotterdam-caas:1.10.7
docker run [OPTIONS] atosclass/rotterdam-caas:1.10.7 [COMMAND] [ARG...]
```
To run Rotterdam in Openshift, deploy the from Docker Hub using the OKD Web UI. The following environment variables can be defined:
- KubernetesEndPoint (e.g.) "http://X.X.X.X:8001⁠"
- OpenshiftEndPoint (e.g.) "https://X.X.X.X:8443⁠"
- ServerIP (e.g.) "X.X.X.X"
- OpenshiftOauthToken (e.g.) "eyJhbGciOiJSUzI1 ... 3MiOiJrdWJlcm5ldGVzL3Nlc"
- SLALiteEndPoint (e.g.) "http://rotterdam-slalite.X.X.X.X.nip.io⁠"
- PrometheusPushgatewayEndPoint (e.g.) "http://pushgateway.X.X.X.X.nip.io⁠"
The SLALiteEndPoint is used to automatically generate SLAs and to stop or terminate them. The SLALite component should also point to Rotterdam to send it the violations.
1. In OKD Web UI, go to selected project / namespace, i.e. the default namespace, and select Add to project > Deploy Image
2. Select Image Name
  - atosclass/rotterdam
    - Name: rotterdam-caasXXX
    - Environment Variables: OpenshiftOauthToken, KubernetesEndPoint, OpenshiftEndPoint, ServerIP, SLALiteEndPoint, PrometheusPushgatewayEndPoint
3. Deploy
4. Go to new application / deplopyment and select Create Route
  - SLALite - Hostname: rotterdam-slalite.192.168.7.28.xip.io
  - Rotterdam - Hostname: rotterdam-cass.192.168.7.28.xip.io

⁠Environment variables

OpenshiftOauthToken: the value of this token has to be created manually after installing Openshift and configuring users and permissions. This variable is supported only when using Openshift. In the case of Kubernetes this variable is not needed.
SLALiteEndPoint: this is the URL of the SLA Manager
PrometheusPushgatewayEndPoint (optional): if there are applications that need to use this tool to push metrics to Prometheus, then the value has to be set
MaxAllowed (optional): Maximun number of violations allowed before sending a notification to the Adaptation Engine. This is a default value used by the platform to decide when to generate a violation in order to take the required actions.
MaxReplicas (optional): Default maximun number of replicas allowed per application. In the case one application needs to be scaled our, this value is used to limit the number of replicas.

⁠Usage Guide

Once Rotterdam is deployed, you can access it through the REST API UI provided by swagger: (e.g. http://rotterdam-caas.X.X.X.X.nip.io/swaggerui/⁠)

The following methods have been defined:

To get information about Rotterdam
- GET /api/v1/version
The following method is used by the SLALite component to send Rotterdam violations and notifications
- POST /api/v1/sla/tasks/{id}/guarantee/{guarantee}
To get all running tasks
- GET /api/v1/tasks
To create a new tasks in Rotterdam
- POST /api/v1/tasks
To view / manage tasks
- GET /api/v1/tasks/{name}
- DELETE /api/v1/tasks/{name}
To get all orchestrators to be managed by Rotterdam
- GET /api/v1/imec
To create a new orchestrators to be managed by Rotterdam
- POST /api/v1/imec
QoS templates
- POST /api/v1/qos/definitions (creates a new QoS template)
- GET /api/v1/qos/definitions
- GET /api/v1/qos/definitions/{name}

⁠Orchestrator definition examples

{
        "ID": "maincluster",
        "Name": "k8s",
        "Description": "main cluster",
        "DefaultDock": "class",
        "Type": "Kubernetes",
        "SO": "ubuntu18",
        "KubernetesEndPoint": "http://X.X.X.X:8001",
        "HostIP": "192.168.7.42",
        "SLALiteEndPoint": "http://rotterdam-slalite.X.X.X.X.nip.io",
        "PrometheusPushgatewayEndPoint": "http://pushgateway.X.X.X.X.nip.io",
        "PrometheusEndPoint": "http://X.X.X.X:32679",
        "User": "vagrant",
        "Password": "vagrant",
        "HostPort": 22
    }

⁠Task definition examples

{
    "name": "adas-my-nginx",
    "dock": "default",
    "cluster": "maincluster",
    "qos": {
      "name": "KubeletTooManyPods",
      "description": "scale down task if cluster pods > 20",
    },
    "replicas": 10,
    "containers": [{
      "name": "adas-my-nginx-app",
      "image": "nginx",
      "ports": [
          {
            "containerPort": "80",
            "hostPort": "80",
            "protocol": "tcp"
          }
        ],
      "volumes": [
        {
          "name": "PATH",
          "mounthPath": "/usr/share/class/vol/"
        }],
			"environment": [
				{
					"name": "TEST_VAL",
					"value":"1.2.3"
				}
			]}
    ]
}

{
  "name": "redis-app-2",
  "dock": "class",
  "qos": {
    "name": "KubeletTooManyPods",
    "description": "scale down task if cluster pods > 20"
  },
  "replicas": 4,
  "containers": [
    {
      "name": "redis",
      "image": "redis",
      "ports": [
        {
          "containerPort": 6379,
          "hostPort": 6379,
          "protocol": "tcp"
        }
      ]
    }
  ]
}

⁠Predefined QoS

QoS templates used to define the SLAs associated to the tasks managed by Rotterdam.

QoS	description	description	source, job
KubeletTooManyPods	infra	scale down task if total of cluster pods is > 20	Prometheus - kubernetes-nodes-cadvisor
KubeletPodsLessThan100	infra	scale down task if total of cluster pods is > 100	Prometheus - kubernetes-nodes-cadvisor
KubeletPodsMoreThan100	infra	scale up task if total of cluster pods is < 100	Prometheus - kubernetes-nodes-cadvisor
Test_Prometheus_01	infra	force a task to be scaled up	Prometheus - kubernetes-nodes-cadvisor

KubeletTooManyPods:

{
    "qos": {
      "name": "KubeletTooManyPods",
      "description": "scale down task if cluster pods > 20"
    }
}

KubeletTooManyPods QoS is equivalent to the following:

"custom": [{
    "guarantees": [{"metric": "kubelet_running_pod_count", "condition": "<", "value": "50"}],
    "action": "scale_down",
    "scalefactor": 2,
    "max": 30,
    "min": 1,
    "type": "infr",
    "name": "cluster_pods_less_50",
    "description": "scale down task if cluster pods > 50"
}]

Test01:

{
    "qos": {
      "name": "Test_Prometheus_01",
      "description": "scale up task if go_memstats_frees_total metric is less than 500000"
    }
}

⁠LICENSE

Rotterdam is licensed under Apache License, version 2⁠.

Tag summary

Recent tags

Content type

Image

Digest

Size

9.6 MB

Last updated

about 5 years ago

docker pull atosclass/rotterdam-caas