A command-line tool to dynamically provision and manage Mesos clusters and their applications. Try it live with Katacoda!
mesosctl was started as a project, Mesosphere's DC/OS was only available for local installations for paying customers.
So, with the creation of
mesosctl the goal was to provide a simple and reliable way of creating and running Mesos clusters on own infrastructure.
In the meantime, DC/OS has been open-sourced, but there assumingly is room for another, although maybe simpler (both in installation, usage and functionality) tool besides DC/OS.
That's why we decided to finish and open-source
mesosctl as well.
mesosctl is running a "sub-shell" application (written as a Node.js module using vorpal.js), meaning it starts a process which keeps the state of the configurations in memory. Unless you
mesosctl shell, you can directly interact with you cluster.
After the cluster has been set up,
mesosctl uses the APIs of the Mesos Masters/Agents, as well as the Marathon API to provide the interactive functionality.
The current version is
0.1.10, which is an early release. We consider it as
alpha and a developer preview.
The preferred ways to run
mesosctl is either via local NPM installation, or via the mesoshq/mesosctl Docker image. Currently, we only
provide a command-line installation. A GUI installer will follow in the future.
mesosctl CLI installation
You can install the NPM package globally by running
npm install mesosctl -g
When installing via the NPM package,
mesosctl expects the following tools to by present on your system:
You can start the Docker image by running
docker run --net=host -it mesoshq/mesosctl mesosctl
If you want to use pre-existing local configurations, please map the relevant folder as a volume like this:
Also, make sure that you host's network settings are configured in a way that the Docker daemon networking can "see" the Vagrant networks.
You can also use a Bash wrapper script, which you should place somewhere in the
PATH, e.g. save the script somewhere and create a symlink to
#!/bin/bash # Create .mesosctl folder in user's home directory is it doesn't exist mkdir -p ~/.mesosctl # Run Docker image and map the local configuration folder docker run --net=host -it -e MESOSCTL_CONFIGURATION_BASE_PATH=/config -v ~/.mesosctl:/config:rw mesoshq/mesosctl mesosctl
Local / Vagrant cluster installation
For testing and local development, one of the provided Vagrant cluster configurations can be used:
- mesoshq/vagrant-cluster-coreos (using CoreOS stable latest)
- mesoshq/vagrant-cluster-ubuntu (using Ubuntu 14.04 latest)
- mesoshq/vagrant-cluster-centos (using CentOS 7 latest)
Obviously, you'll need a working Vagrant installation (>= v1.8, with VirtualBox).
You can use
mesosctl to install Mesos clusters on freshly installed Linux machines. It currently supports the following OS families/flavors:
- Debian Jessie
- Ubuntu Xenial
- Ubuntu Vivid
- Ubuntu Trusty
- CentOS 7
- CentOS 6
- RHEL 7
- RHEL 6
The requirements on the remote machines which should be utilized by
mesosctl are the following:
- A common user with sufficient permission to install packages and run services, as well as the belonging SSH key of this user to be able to connect via SSH. This SSH key needs to be present on the machine you're running
- A Python version >= 2.4 (but if you are running less than Python 2.5 on the remotes, you will also need
- Network connectivity to see the other hosts of the cluster
- Internet access (installer packages and Docker images are downloaded during installation)
The remote machines need to be reachable from the host you're running
mesosctl on, otherwise the installation will fail.
Using cloud providers
Currently, either the direct access via SSH is supported, or via a preconfigured and running "SSH over VPN" connection.
See the following links on how to configure SSH access for cloud-hosted nodes:
Once you have installed
mesosctl either as global NPM module or as Docker container (having the wrapper script in the path), you can use
to open the
mesosctl sub-shell application.
Create a configuration file
Configuration files for
mesosctl are YAML files. To be able to provision a Mesos cluster, you'll need to specify the following details:
cluster_name: The desired cluster name. Must be a
stringcontaing no spaces.
os_family: The OS family. Must be one of
RedHat(see Remote installation).
ssh_key_path: The (absolute) path to the SSH key which provides access to all hosts. This must be the same for all hosts.
ssh_user: The username belonging to the SSH key specified above. This must be the same for all hosts.
ssh_port: The SSH port to access all hosts. This must be the same for all hosts.
dns_servers: The list of external DNS resolvers.
masters: The list of IP addresses which should be provisioned as Mesos masters.
agents: The list of IP addresses which should be provisioned as Mesos agents.
registry: The IP address which should be provisioned as private Docker registry.
An example configuration file is the following:
cluster_name: my_first_mesos_cluster os_family: Debian ssh_key_path: /Users/developer/.vagrant.d/insecure_private_key ssh_port: 22 ssh_user: vagrant dns_servers: - 220.127.116.11 - 18.104.22.168 masters: - 172.17.10.101 - 172.17.10.102 - 172.17.10.103 agents: - 172.17.10.101 - 172.17.10.102 - 172.17.10.103 registry: - 172.17.10.101
Using pre-existing configurations
If you have a pre-existing configuration file (e.g. when you use a Vagrant cluster setup), you can use the following command to load it:
mesosctl $ config load /path/to/mesosctl.yml
Create a configuration via
Once you have a configuation, you can run
mesosctl $ config validate --> The current configuration is valid!
If the configuration is valid, then you can start with the provisioning of the cluster:
mesosctl $ cluster provision --verbose
This may take a while, because
mesosctl will trigger the installation of all required software on the configured cluster hosts. If you used one of the Vagrant sample clusters (see above), once the provisioning of the cluster is finished, you'll have a 3 node cluster, where each node is running the Mesos Master, the Mesos Agent and a Marathon instance. The respective IP addresses can also be found in the Vagrant project's docs.
You can check the cluster status via
mesosctl like this:
mesosctl $ cluster status Cluster 'mesos_cluster_ubuntu' utilization: CPU [== ] 10% Memory [= ] 6% Disk [ ] 0% Ports [ ] 0%
You can also retrieve the current leader's address, which you can use in a browser to get the Mesos Master's UI:
mesosctl $ cluster get leader --> Current leading Master's address is 172.17.10.103:5050
You can use the following commands with
You can create, update and load existing configurations.
config create Creates a configuration config load [pathToConfig] Loads an existing configuration, either from specified path or from a selection of existing configurations config show Displays the current configuration config validate Validates the current configuration config get clustername Gets the cluster name config set clustername <clusterName> Defines the cluster name config set os Defines the OS type config get os Gets the OS type config set ssh.keypath <path> Defines the path to the SSH key for accessing the hosts config get ssh.keypath Gets the path to the SSH key for accessing the hosts config set ssh.user <userName> Defines the user name for the SSH key for accessing the hosts config get ssh.user Gets the user name for the SSH key for accessing the hosts config set ssh.port <port> Defines the port for the SSH connection for accessing the hosts config get ssh.port Gets the port for the SSH connection for accessing the hosts config set admin.user <userName> Defines the admin user name config get admin.user Gets the admin user name config set admin.password <password> Defines the admin password config set dns.servers [dnsServer...] Defines the DNS nameservers config add dns.servers [dnsServer...] Adds IP address(es) to the DNS nameserver list config remove dns.servers [dnsServer...] Remove IP address(es) from the DNS nameserver list config set masters [masterServer...] Defines the Mesos Master servers config add masters [masterServer...] Adds IP address(es) to the Mesos Master server list config remove masters [masterServer...] Remove IP address(es) from the Mesos Master servers list config set agents [agentServer...] Defines the Mesos Agent servers config add agents [agentServer...] Adds IP address(es) to the Mesos Agent server list config remove agents [agentServer...] Remove IP address(es) from the Mesos Agent servers list config set registry <registryServer> Defines the private Docker Registry server config get registry Gets the private Docker Registry server IP address
You can provison a cluster based on the before defined configuration, and get status information as well as connect to nodes via SSH.
cluster provision [options] Provisions the cluster based on the current configuration cluster status Display the cluster status cluster status agent <agentIPAddress> Display the Mesos agent status and utilization cluster ssh <ipAddress> [command] Issue a SSH command on the remote host. The command must be wrapped in double quotation marks, like `"ls -la"`. cluster get leader Returns the currently leading Mesos Master's IP address cluster restart services Restarts the ZooKeeper, Master, Agent and Marathon services on both Masters and Agents
You can use the standard DC/OS (Mesosphere) universe repository.
repository install Download and installs the current DC/OS Universe repository repository update Updates the local DC/OS Universe repository with the remote repository check Checks if the DC/OS Universe repository is installed locally
You (un)install packages from the DC/OS universe repository. Please be aware that
mesosctl cannot programatically check whether the actual configuration supports the chosen package from a resource perspective (number of nodes, available cpus and memory), because this information is currently not exposed in the package definitions.
package install [options] <packageName> Installs a package package describe [options] <packageName> Displays information about a package package uninstall <packageName> Uninstalls a package package search <searchString> Searches for packages with specific string
Marathon applications and groups
You can start and manage applications/groups via Marathon.
marathon info list Shows information about the running Marathon instance marathon app list Lists all running apps marathon app remove <appId> Removes a specific app (i.e. stops the app) marathon app restart <appId> Restarts a specific app marathon app show <appId> Show the configuration details of a specific app marathon app start <pathToJSON> Starts an app with a specific configuration marathon app update <appId> [properties...] Updates a running specific app marathon app version list <appId> Display the version list for a specific app marathon app scale <appId> <instances> Scales (up od down) a specific app marathon deployment list Lists all current deployments marathon deployment rollback <deploymentId> Triggers a rollback of a specific deployment marathon deployment remove <deploymentId> Removes/stops a specific deployment marathon group add <pathToJSON> Adds a new group marathon group list Lists all current groups marathon group scale <groupId> <instances> Scales (up or down) a specifc group marathon group show <groupId> Show details about a specific group marathon group remove <groupId> Remove/stop a specific group marathon task list Show all running tasks marathon task show <taskId> Show details about a specify running task
You can introspect running tasks from the leading Mesos master.
task list [options] Lists all task in the cluster task show [options] <task> Retrieves information about a task task log [options] <taskId> [file] Print the task log. By default, the 10 most recent task logs from stdout are printed. task ls <taskId> [path] Print the list of files in the Mesos task sandbox