absaoss/spline-spark-agent repository overview

Data Lineage Tracking And Visualization Solution

Spline project aims to help people and organizations to get insight into data processing performed by Apache Spark™ or other data transformation frameworks.

⁠Spline agent for Apache Spark with examples

This repository contains a Spline Spark agent along with a set of examples that are run automatically when the container starts up. The main purpose of it is populating a newly created Spline server with a sample data. When example jobs are finished the container shuts down.

To run examples you need a Spline REST server⁠ ready.

⁠Usage

docker run --rm -e SPLINE_PRODUCER_URL=... absaoss/spline-spark-agent

The correct Producer API URL can be found on the index page of your running Spine REST Server.

For more information see the Spline project GitHub pages⁠

⁠Environment variables

⁠Spline settings

⁠HTTP proxy settings

These are standard Java HTTP Proxy settings⁠:

Copyright 2019 ABSA Group Limited

you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.