Public | Automated Build

Last pushed: 4 months ago
Short Description
Podded webui for launching Apache Spark clusters on OpenShift
Full Description


oshinko-webui

This project provides a solution for deploying and managing Apache Spark
clusters in an OpenShift environment. The oshinko-webui is deployed into a
project within OpenShift, and then can create, update, and destroy Apache
Spark clusters in that project. Once installed, it consists of a Node.JS
application that is contained within a Pod and provides a web browser based
user interface for controlling the lifecycle of Spark clusters.

Installation

In general, there are two main audiences for a discussion of installing the
oshinko-webui: users and developers. If you are interested in running as a
user, or to test drive the application, please see the
Step-by-step quickstart
instructions. If you would like to get started hacking on oshinko-webui please
see the
Developer instructions
section.

Step-by-step quickstart

These instructions assume that you have access to an OpenShift cluster and
the oc command line tool. Although these instructions will help you to
install the oshinko-webui into your OpenShift project, it is possible in
some circumstances that you will not have enough privileges to run the
installation. In the event that you are unable to create the necessary
components to install the application, please consult with your OpenShift
administrator.

Before performing the following instructions, you must be logged in to
your account and project using the oc tool.

Step 1. Create the service account and template

For oshinko-webui to interact with OpenShift and control the Spark resources
you will create, it needs a service account in your project. The service account
is created with edit permissions along with the oshinko-webui template by issuing
the following command:

oc create -f https://raw.githubusercontent.com/radanalyticsio/oshinko-webui/master/tools/ui-template.yaml

Step 2. Run oshinko-webui

oc new-app --template=oshinko-webui

Developer instructions

If you are interested in developing the code for oshinko-webui or hacking on
its internals, the following instructions will help you to deploy, run, and
test the code.

Before getting started you will need to have access to an OpenShift cluster,
the oc command line application, and the
oshinko-cli command line
application.

Running the app during development

You'll need to have a node environment installed (developed using NodeJS v6.3.1).
You might prefer to use nvm (https://github.com/creationix/nvm)
to manage your node environment.
Once that is set up, you can run the following:

$ npm install
$ npm install -g bower
$ bower install

Now you're ready to run the oshinko-webui server.

Note, a working local "oc" binary is expected.
To run locally, you'll need a proxy to the api server running.
The following will run a basic proxy, see oc proxy --help if you require
something more specific.

$ oc proxy --disable-filter=true --api-prefix=/proxy &

Edit the exports in scripts/launch-local.sh to match your environment.

Change to the scripts directory and run

$ ./launch-local.sh

You can pick one of these options:

  • install node.js and run node server.js

Then navigate your browser to http://localhost:<port> to see the app running in
your browser.

Running unit tests

To run the unit tests:

$ npm install -g karma-cli
$ karma start test/karma.conf.js

End to end testing

The end to end tests can be run using the test/e2e.sh script.
This script assumes a current login to an OpenShift instance
and it runs the test in the current project (it's recommended
to create a fresh project for the test run). It also assumes
that the local oshinko-webui repository has been setup (ie all
the dependencies have been installed and the webui components
have been installed with npm and bower as noted above).

The test/e2e.sh script will create a serviceaccount,
templates, a configmap, etc in the current project as part
of the test.

As a convenience, the test-e2e and test-e2e-secure make
targets can be used to run the test. These targets will
first build a local image and then run the test with defaults.
For example:

$ oc new-project mywebuitest
$ make test-e2e
...
$
$ oc new-project mysecuretest
$ make test-e2e-secure
...

The environment variables below can be set for the call
to make, for example:

$ WEBUI_START_XVFB=false make test-e2e

Environment variables for test configuration

There are several enviroment variables that you can set
to configure the tests:

WEBUI_START_XVFB (default is true)

This causes the test to start an Xvfb server running
for display 99 if it's not already running (required).

WEBUI_TEST_IMAGE (default is oshinko-webui:latest if WEBUI_TEST_LOCAL_IMAGE is true or docker.io/radanalyticsio/oshinko-webui otherwise)

The image to use for testing. The defaults are set up to
reference an image from the local docker host (ie, one that has just been
built) but this setting can be used to reference an image from an arbitrary
docker registry.

WEBUI_TEST_LOCAL_IMAGE (default is true)

This indicates that the s2i images to be tested are local, that is they
are available from the local docker daemon but not in an external registry
like docker hub.

If this is set to "false", the test image is assumed to be in an external
registy. WEBUI_TEST_INTEGRATED_REGISTRY and WEBUI_TEST_EXTERNAL_REGISTRY
will be ignored because there will be no need to push local images to
a registry.

WEBUI_TEST_INTEGRATED_REGISTRY

This is the IP address of the integrated registry. Use this setting when:

  • running the test using local images
  • running the test on a host where the integrated registry is reachable (like the OpenShift master)
  • using an OpenShift instance that was not created with oc cluster up
$ WEBUI_TEST_INTEGRATED_REGISTRY=172.123.456.89:5000 test/e2e.sh

WEBUI_TEST_EXTERNAL_REGISTRY

This is the IP address of a docker registry. If this is set then
WEBUI_TEST_EXTERNAL_USER and WEBUI_TEST_EXTERNAL_PASSWORD must also
be set so that the tests can log in to the registry.
Use this setting when:

  • running the test using local images
  • running the test from a host where the integrated registry is not reachable
  • using an OpenShift instance that was not created with oc cluster up

WEBUI_TEST_SECURE (default is false)

Use the template for a secure webui. If this is set to true
and the OpenShift instance was not created with oc cluster up,
then WEBUI_TEST_SECURE_USER and WEBUI_TEST_SECURE_PASSWORD
should be used to set login credentials for the webui.

WEBUI_TEST_SECURE_USER (default is "developer")

Username to use for a secure webui test

WEBUI_TEST_SECURE_PASSWORD (default is "deverloperpass")

Password to use for a secure webui test

WEBUI_TEST_RESOURCES (default is local tools/resources.yaml)

The resources.yaml file used to set up test resources. The value
may be a file path, or it may be a url such as
https://radanalytics.io/resources.yaml.

Dependencies for end to end tests

The end to end tests require a number of dependencies.
The test/e2e-setup.sh script has been provided to install
the dependencies and setup an oshinko-webui repository
for testing. This is especially helpful when setting up
a clean machine.

Note, test/e2e-setup.sh assumes passwordless sudo.

You can look through the script and see what it installs
and make sure those things are installed yourself or you
can do this:

$ test/e2e-setup.sh  # from the oshinko-webui main directory

The script should be idempotent.

Docker Pull Command
Owner
crobby
Source Repository