Introducing our new CEO Don Johnson - Read More

mariadb/columnstore

Verified Publisher

By MariaDB Corporation

Updated over 1 year ago

MariaDB ColumnStore Engine for Analytics

Image
4

50K+

MariaDB

Columnstore Docker Project

Summary

MariaDB ColumnStore is a columnar storage engine that utilizes a massively parallel distributed data architecture. It was built by porting InfiniDB to MariaDB and has been released under the GPL license.

MariaDB ColumnStore is designed for big data scaling to process petabytes of data, linear scalability and exceptional performance with real-time response to analytical queries. It leverages the I/O benefits of columnar storage, compression, just-in-time projection, and horizontal and vertical partitioning to deliver tremendous performance when analyzing large data sets.

Requirements

Please install the following software packages before you begin.

Docker Run Instructions (Single Node)

docker run -d -p 3307:3306 --shm-size=512m -e PM1=mcs1 --hostname=mcs1 --name mcs1 mariadb/columnstore
docker exec -it mcs1 provision mcs1
Waiting for PM1 To Be Initialized .. done
Adding PM(s) To Cluster ... done
Restarting Cluster ... done
Validating ColumnStore Engine ... done

Run Variables

VariableTypeDefaultRequired
ADMIN_HOSTString%No
ADMIN_PASSStringC0lumnStore!No
ADMIN_USERStringAdminNo
CEJ_PASSStringC0lumnStore!No
CEJ_USERStringcejNo
CGROUPString./Yes
CMAPI_KEYStringsomekey123No
PM1Hostname-Yes
S3_ACCESS_KEY_IDStringNoneNo
S3_BUCKETStringNoneNo
S3_ENDPOINTURLNoneNo
S3_REGIONStringNoneNo
S3_SECRET_ACCESS_KEYStringNoneNo
USE_S3_STORAGEBooleanfalseNo

Docker Compose Instructions (Cluster)

git clone https://github.com/mariadb-corporation/mariadb-columnstore-docker
cd mariadb-columnstore-docker
cp .env_example .env
  • Edit .env with your custom settings
docker compose up -d
docker exec -it mcs1 provision mcs1 mcs2 mcs3
Waiting for PM1 To Be Initialized .... done
Adding PM(s) To Cluster ... done
Restarting Cluster ... done
Validating ColumnStore Engine ... done

Custom Build Instructions (Optional)

git clone https://github.com/mariadb-corporation/mariadb-columnstore-docker
cd mariadb-columnstore-docker
cp .env_example .env
cp .secrets_example .secrets
./build

Access

Database Access

mysql -h 127.0.0.1 -P 3307 -u admin -p

The default password is: C0lumnStore!

MaxScale 1 GUI Access

  • URL: http://127.0.0.1:8989
  • username: admin
  • password: mariadb

MaxScale 2 GUI Access

  • URL: http://127.0.0.1:8990
  • username: admin
  • password: mariadb

Glossary Items

  • PM: Performance Module
  • PM1: Primary Database Node
  • PM2: Secondary Database Node
  • PM3: Tertiary Database Node
  • MX1: Primary MaxScale Node
  • MX2: Secondary MaxScale Node

CLI Instructions

Set API Code:
mcs cluster set api-key --key <api_key>

Get Status:

mcs cluster status

Start Cluster:

mcs cluster start

Stop Cluster:

mcs cluster stop

Add Node:

mcs cluster node add --node <node>

Remove Node:

mcs cluster node remove --node <node>

Mode Set Read Only:

mcs cluster set mode --mode readonly

Mode Set Read/Write:

mcs cluster set mode --mode readwrite

Log Info

Logs are stored in /var/log/mariadb/columnstore

They are also available via the following command:

docker logs mcs1

Simple Backup Script Example

#!/bin/bash
rm -rf /backup
mkdir -p /backup/{mysql,columnstore}
mariadb-backup --user=root --backup --rsync --target-dir=/backup/mysql
mcs cluster set mode --mode readonly
rsync -av /var/lib/columnstore/ /backup/columnstore/
mcs cluster set mode --mode readwrite

Docker Pull Command

docker pull mariadb/columnstore