graphistry/graph-app-kit-st repository overview

⁠ ⁠ ⁠ ⁠ ✔️ Linux ✔️ OS X ❌ Windows (#39⁠)

⁠Welcome to graph-app-kit

Turn your graph data into a secure and interactive visual graph app in 15 minutes!

⁠Why

This open source effort puts together patterns the Graphistry team has reused across many graph projects as teams go from code-heavy Jupyter notebook experiments to deploying streamlined analyst tools. Whether building your first graph app, trying an idea, or wanting to check a reference, this project aims to simplify that process. It covers pieces like: Easy code editing and deployment, a project stucture ready for teams, built-in authentication, no need for custom JS/CSS at the start, batteries-included data + library dependencies, and fast loading & visualization of large graphs.

⁠What

• Minimal core: The barebones dashboard server. In provides a StreamLit docker-compose container with PyData ecosystem libraries and examples of visualizing data from various systems. Install it, plug in credentials to various web services like cloud databases and a free Graphistry Hub⁠ visualization account, and launch. It does not have GPU ETL and GPU AI libraries.

• Full core: Initially for AWS, the full core bundles adds to the docker-compose system: Accounts, Jupyter notebooks for authoring, serves StreamLit dashboards with both public + private zones, and runs Graphistry/RAPIDS locally on the same server. Launch with on click via the Cloud Formation template.

• Full core + DB: DB-specific variants are the same as minimal/full, and add simpler DB-specific quick launching/connecting.

⁠Get started

⁠Quick (Local code) - full GPU core + third-party connectors

Note: Base image includes Nvidia RAPIDS and AI dependencies so is quite large, see CPU alternative for a lightweight alternativve

Note: Use sudo for docker-compose commands if your configuration requires it and is giving permission error

# Minimal core
git clone https://github.com/graphistry/graph-app-kit.git
cd graph-app-kit/src/docker

# Enable docker buildkit
# ... or run docker-compose via provided alias script `./dc`
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1

# Build
docker-compose build

# Optional: Edit src/docker/.env (API accounts), docker-compose.yml: Auth, ports, ...

# Launch
docker-compose up -d
docker-compose logs -f -t --tail=100

=> http://localhost:8501/

To add views⁠ and relaunch:

# Add dashboards @ src/python/views/<your_custom_view>/__init__.py

docker-compose up -d --force-recreate

⁠Quick (Local code) - minimal CPU core + third-party connectors

Same commands as above, but use ./dc.cpu, which aliases docker-compose -f docker-compose.yml -f override/cpu.override.yml:

git clone https://github.com/graphistry/graph-app-kit.git
cd graph-app-kit/src/docker
./dc.cpu build
...
./dc.cpu up

⁠Quick Launchers - minimal/full core

1. Quick launch options:

Full:

• Public + protected Streamlit dashboards, Jupyter notebooks + editing, Graphistry, RAPIDS
• Login to web UI as admin / i-instanceid -> file uploader, notebooks, ...
• Dashboards: /public/dash and /private/dash
• More info⁠

Admin:

# launch logs
tail -f /var/log/cloud-init-output.log -n 1000

# app logs
sudo docker ps
sudo docker logs -f -t --tail=1 MY_CONTAINER

# restart a graphistry container
cd graphistry && sudo docker-compose restart MY_CONTAINER

# restart caddy (Caddy 1 override)
cd graphistry && sudo docker-compose -f docker-compose.gak.graphistry.yml up -d caddy

# run streamlit
cd graph-app-kit/public/graph-app-kit && docker-compose -p pub run -d --name streamlit-pub streamlit
cd graph-app-kit/private/graph-app-kit && docker-compose -p priv run -d --name streamlit-priv streamlit

Minimal: Open Streamlit, ssh to connect/add free Graphistry Hub username/pass⁠:

Database-specific: Amazon Neptune⁠, TigerGraph⁠

1. Add views⁠

2. Main configurations and extensions⁠: Database connectors, authentication, notebook-based editing, and more

⁠The pieces

⁠Core

• Prebuilt Python project structure ready for prototyping

• Streamlit⁠ quick self-serve dashboarding

• Graphistry⁠ point-and-click GPU-accelerated visual graph analytics

• Data frames: Data wrangling via Pandas⁠and Apache Arrow⁠, including handling formats such as CSV, XLS, JSON, Parquet, and more

• Standard Docker and docker-compose cross-platform deployment

⁠GPU acceleration (optional) - Full

For non-minimal installs, if GPUs are present, graph-app-kit leverages GPU cloud acceleration:

• GPU Analytics: RAPIDS⁠ and CUDA already setup for use if run with an Nvidia docker runtime - cudf GPU dataframes, BlazingSQL⁠ GPU SQL, cuGraph GPU graph algorithms, cuML libraries, and more

• GPU Visualization: Connect to an external Graphistry server or, faster, run on the same GPU server

⁠Prebuilt integrations & recipes

graph-app-kit works well with the Python data ecosystem (pandas, cudf, PySpark, SQL, ...) and we're growing the set of builtins and recipes:

• Graph databases

  • AWS Neptune⁠: quick launch⁠ and manual launch⁠

  • TinkerPop Gremlin⁠: query demos⁠

  • TigerGraph⁠: setup⁠

  • Collaborations welcome!

• Jupyter notebooks⁠: Use quick launchers or integrations guide⁠ for web-based live editing of dashboards by sharing volume mounts between Jupyter and Streamlit

• Caddy⁠: Reverse proxy for custom URLs, automatic LetsEncrypt TLS certificates⁠, multiple sites on the same domain, pluggable authentication (see integrations guide⁠)

⁠Contribute

We welcome all sorts of help!

• Deployment: Docker, cloud runners, ...
• Dependencies: Common graph packages
• Connectors: Examples for common databases and how to get a lot of data out
• Demos!

See develop.md⁠ for more contributor information