Sign inSign up

jonhartley/mysql-cluster8-sql

By jonhartley

Updated about 7 years ago

NDB V8 for Docker/Kubernetes

Image
0

1.5K

jonhartley/mysql-cluster8-sql repository overview

MySQL Community Cluster SQL Node

Basic Info
  • This is one of the containers for building a MySQL Community Cluster (NDB Engine) in Docker or Kubernetes
  • See jonhartley/mysql-cluster-data jonhartley/mysql-cluster-mgm and jonhartley/mysql-cluster-memcached
ENV Variables for this contianer
  • MGM = "mgm-0" -- mangement nodes (comma separated) or service name
  • CONFIG="max_allowed_packet=1G,innodb_buffer_pool_size=32M" -- comma separated extra config for MySQL
  • MYSQL_ROOT_PASSWORD=password -- roots password a global root user will aslo be added
  • DB="feed" -- create a schema on start
  • SQL="/var/sql/add_func.sql" -- add sql on start can include users, functions, tables etc.
  • MONITOR="monitor:982347jhbciuy2398yu" -- add a monitor user for telegraf

docker network create clusternet

docker run --net clusternet \
--name data-0 \
--hostname data-0 \
-e MGM="mgm-0" \
-e MT="false" \
-d jonhartley/mysql-cluster-data

docker run --net clusternet \
--name data-1 \
--hostname data-1 \
-e MGM="mgm-0" \
-e MT="false" \
-d jonhartley/mysql-cluster-data

  • Management Node
docker run --net clusternet \
--name mgm-0 \
--hostname mgm-0 \
-e HAMGM="false" \
-e DATANODES="2" \
-e DATARES="data-0" \
-e API=10 \
-e DATAMEM="512M" \
-e NOREP="2" \
-d jonhartley/mysql-cluster-mgm
  • Once management node is up
docker exec -it mgm-0 bash
bash-4.3#>ndb_mgm -c localhost
-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=1  @172.18.0.3  (mysql-5.7.23 ndb-7.6.7, not started)
id=2  @172.18.0.4  (mysql-5.7.23 ndb-7.6.7, not started)

[ndb_mgmd(MGM)] 1 node(s)
id=51 @172.18.0.2  (mysql-5.7.23 ndb-7.6.7)

[mysqld(API)] 10 node(s)
id=61 (not connected, accepting connect from any host)
id=62 (not connected, accepting connect from any host)
id=63 (not connected, accepting connect from any host)
id=64 (not connected, accepting connect from any host)
id=65 (not connected, accepting connect from any host)
id=66 (not connected, accepting connect from any host)
id=67 (not connected, accepting connect from any host)
id=68 (not connected, accepting connect from any host)
id=69 (not connected, accepting connect from any host)
id=70 (not connected, accepting connect from any host)

ndb_mgm>ALL START
NDB Cluster is being started.

Node 1: Start initiated (version 7.6.7)
ndb_mgm> Node 2: Start initiated (version 7.6.7)
Node 1: Started (version 7.6.7)
Node 2: Started (version 7.6.7)

ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=1  @172.18.0.3  (mysql-5.7.23 ndb-7.6.7, Nodegroup: 0, *)
id=2  @172.18.0.4  (mysql-5.7.23 ndb-7.6.7, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=51 @172.18.0.2  (mysql-5.7.23 ndb-7.6.7)

  • Once the cluster is started add an SQL node
docker run --net clusternet \
--name sql-0 \
--hostname sql-0 \
-e MGM="mgm-0" \
-e CONFIG="max_allowed_packet=1G,innodb_buffer_pool_size=32M" \
-e MYSQL_ROOT_PASSWORD=password \
-d jonhartley/mysql-cluster-sql
  • On the SQL node is a small script /postinstall.sh, this will allow you to enable centralized grants and MemCached API
  • Then you can add a Memcached API node
docker run --net clusternet \
--name mc-0 \
--hostname mc-0 \
-e MGM="mgm-0" \
-e STARTOPTS="'connectstring=mgm-0;role=db-only'" \
-e MCTHREADS="2" \
-e MCCONNS="20" \
-d jonhartley/mysql-cluster-memcached
Kubernetes
  • Kubenetes installation must have DNS for services enabled
  • Adjust rousources as required - DataMemory is set in the Management container, ensure they are compatible
  • Below is a StateFull set with Cinder PVs for cluster datadir and an NFS mount for backups

apiVersion: v1
kind: Service
metadata:
  name: ndbd
  namespace: mysqlcluster
  labels:
    app: datanode
spec:
  ports:
  - port: 2202
    name: ndbd
  clusterIP: "None"
  selector:
    app: datanode
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: ndbd-pdb
  namespace: mysqlcluster
spec:
  selector:
    matchLabels:
      app: datanode
  maxUnavailable: 1
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: data
  namespace: mysqlcluster
  labels:
    app: datanode
spec:
  serviceName: ndbd
  replicas: 4
  podManagementPolicy: OrderedReady
  updateStrategy:
      type: OnDelete
  template:
    metadata:
      labels:
        app: datanode
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values: 
                    - datanode
              topologyKey: "kubernetes.io/hostname"
      terminationGracePeriodSeconds: 300
      containers:
      - name: k8sndbd
        image: jonhartley/mysql-cluster-data
        imagePullPolicy: Always
        ports:
        - containerPort: 2202
          name: ndbd
        resources:
          requests:
            memory: "1700Mi"
            cpu: "600m"
          limits:
            memory: "1700Mi"
            cpu: "600m"
        env:
        - name: MGM
          value: "mgm"
        - name: MT 
          value: "false"
        readinessProbe:
          exec:
            command:
              - /usr/bin/pgrep 
              - ndbd
          initialDelaySeconds: 5
          periodSeconds: 15
        volumeMounts:
          - name: mcnfs
            mountPath: "/var/lib/mysqlcluster/backup"
            subPath: clusterbackup
          - name: data
            mountPath: "/var/lib/mysqlcluster/data"
      volumes:
        - name: mcnfs
          persistentVolumeClaim:
              claimName: mcnfs
        - name: data
          persistentVolumeClaim:
              claimName: data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "cinder"
      resources:
        requests:
          storage: 10Gi

  • Management nodes are alos deployed in a StateFulSet
  • Again using small Cinder volumes for state of cluster and allowing these nodes to access the backup storage for restore
apiVersion: v1
kind: Service
metadata:
  name: mgm
  namespace: mysqlcluster
spec:
  ports:
    - port: 1186
      name: ndbmgmd
  clusterIP: "None"
  selector:
    app: mgmnode
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mgm
  namespace: mysqlcluster
  labels:
    app: mgmnode
spec:
  serviceName: mgm
  replicas: 2
  podManagementPolicy: Parallel
  updateStrategy:
      type: OnDelete
  template:
    metadata:
      labels:
        app: mgmnode
    spec:
      tolerations:
      - key: "key"
        operator: "Exists"
        effect: "NoSchedule"
      nodeSelector:
        location: master
      containers:
        - resources:
            limits:
              memory: "100Mi"
              cpu: "100m"
            requests:
              memory: "100Mi"
              cpu: "100m"
          image: jonhartley/mysql-cluster-mgm
          name: mgm
          ports:
            - containerPort: 1186
              name: ndbmgm
          volumeMounts:
            - name: mcnfs
              mountPath: "/var/lib/mysqlcluster/backup"
              subPath: clusterbackup
            - name: config
              mountPath: "/var/lib/mysqlcluster/mgm"
          imagePullPolicy: Always
          env:
            - name: HAMGM
              value: "true"
            - name: DATANODES
              value: "4"
            - name: DATARES
              value: "data-0.ndbd"
            - name: API
              value: "10"
            - name: DATAMEM
              value: "756M"
            - name: NOREP
              value: "2"
            - name: MAXATT
              value: "5000"
            - name: MAXCO
              value: "40000"
      readinessProbe:
        exec:
          command:
            - cat 
            - /var/ready
        initialDelaySeconds: 5
        periodSeconds: 20
      livenessProbe:
        tcpSocket:
          port: 1186
        initialDelaySeconds: 15
        periodSeconds: 20
      volumes:
        - name: mcnfs
          persistentVolumeClaim:
              claimName: mcnfs
        - name: config
          persistentVolumeClaim:
              claimName: config
  volumeClaimTemplates:
  - metadata:
      name: config
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "cinder"
      resources:
        requests:
          storage: 5Gi
  • SQL Nodes can be deployed in the same way
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: sqlnode
  namespace: mysqlcluster
  labels:
    app: sqlnode
spec:
  serviceName: sqlnode
  replicas: 1
  podManagementPolicy: Parallel
  updateStrategy:
      type: RollingUpdate
  template:
    metadata:
      annotations:
        prometheus.io/port: "9104"
        prometheus.io/scrape: "true"
      labels:
        app: sqlnode
    spec:
      containers:
        - resources:
            limits:
              memory: "400Mi"
              cpu: "500m"
            requests:
              memory: "400Mi"
              cpu: "500m"
          image: jonhartley/mysql-cluster-sql
          name: sqlnode
          ports:
            - containerPort: 3306
          volumeMounts:
            - name: mcnfs
              mountPath: "/var/lib/mysqlcluster/backup"
              subPath: clusterbackup
          imagePullPolicy: Always
          env:
            - name: MGM
              value: "mgm"
            - name: CONFIG
              value: "max_allowed_packet=1G,innodb_buffer_pool_size=32M"
            - name: MYSQL_ROOT_PASSWORD
              value: "password"
      readinessProbe:
        tcpSocket:
          port: 3306
        initialDelaySeconds: 5
        periodSeconds: 10
      livenessProbe:
        tcpSocket:
          port: 3306
        initialDelaySeconds: 15
        periodSeconds: 20
      volumes:
        - name: mcnfs
          persistentVolumeClaim:
              claimName: mcnfs
---
apiVersion: v1
kind: Service
metadata:
  name: sqlnode
  namespace: mysqlcluster
  labels:
    app: sqlnode
spec:
  type: NodePort
  ports:
    - port: 3306
      name: mysql
      nodePort: 32306
  clusterIP: "10.106.1.10"
  selector:
    app: sqlnode
---
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: sqlnode
  namespace: mysqlcluster
spec:
  scaleTargetRef:
    apiVersion: apps/v1beta1
    kind: StatefulSet
    name: sqlnode
  maxReplicas: 4
  minReplicas: 2
  targetCPUUtilizationPercentage: 50

Tag summary

Content type

Image

Digest

Size

948.4 MB

Last updated

about 7 years ago

docker pull jonhartley/mysql-cluster8-sql