Sign inSign up

sanglr/datax_web

By sanglr

•Updated almost 4 years ago

1.0版本为datax202210版本

Image
0

10K+

sanglr/datax_web repository overview

目录 datax是阿里开发的一款数据集成工具。因工作中有用到，将安装和配置过程分享给大家。

部署 操作系统为CentOS Linux release 7.9.2009 (Core)

⁠1，安装mysql数据库，版本为5.7

sudo yum localinstall https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm
sudo yum install mysql-community-server

按照步骤2的配置文件bootstrap.properties，将用户创建好并赋予正确的权限。

登录mysql，创建需要的数据库，

create database datax_web_db default character set utf8mb4 collate utf8mb4_general_ci;
grant all privileges on datax_web_db.* to 'datax_web'@'%';
flush privileges;

导入datax_web所需的表，下载 datax_web.sql⁠，在mysql命令行执行下面命令，

use datax_web_db;
source /tmp/datax_web.sql;

⁠2，datax_web配置部署

docker 命令

docker run -itd --name datax_web -p 9527:9527 \
--restart=always \
-v /data/docker/datax/config/bootstrap.properties:/home/datax/datax-web-2.1.2/modules/datax-admin/conf/bootstrap.properties \
-v /data/docker/datax/config/core.json:/home/datax/datax/conf/core.json sanglr/datax_web:3.0.1

采用docker方式运行，使用镜像“linshellfeng/datax_web:3.0.1”其包含datax和datax_web， docker-compose.yaml

version: '3.3'
services:
  datax_web:
    image: sanglr/datax_web:3.0.1
    ports:
      - "9527:9527"
    restart: always
    volumes:
      - "./config/bootstrap.properties:/home/datax/datax-web-2.1.2/modules/datax-admin/conf/bootstrap.properties"
      - "./config/core.json:/home/datax/datax/conf/core.json"
⁠config/bootstrap.properties，根据自己的实际情况填写
#Database

#数据库所在ip
DB_HOST=192.168.52.128

#数据库端口号
DB_PORT=3306

#数据库用户名，第一步创建的用户
DB_USERNAME=datax_web

#数据库用户密码，第一步创建的用户密码
DB_PASSWORD=password

#数据库名称
DB_DATABASE=datax_web_db
⁠config/core.json
{
    "entry": {
        "jvm": "-Xms1G -Xmx1G",
        "environment": {}
    },
    "common": {
        "column": {
            "datetimeFormat": "yyyy-MM-dd HH:mm:ss",
            "timeFormat": "HH:mm:ss",
            "dateFormat": "yyyy-MM-dd",
            "extraFormats":["yyyyMMdd"],
            "timeZone": "GMT+8",
            "encoding": "utf-8"
        }
    },
    "core": {
        "dataXServer": {
            "address": "http://localhost:7001/api",
            "timeout": 10000,
            "reportDataxLog": false,
            "reportPerfLog": false
        },
        "transport": {
            "channel": {
                "class": "com.alibaba.datax.core.transport.channel.memory.MemoryChannel",
                "speed": {
                    "byte": 2000000,
                    "record": -1
                },
                "flowControlInterval": 20,
                "capacity": 512,
                "byteCapacity": 67108864
            },
            "exchanger": {
                "class": "com.alibaba.datax.core.plugin.BufferedRecordExchanger",
                "bufferSize": 32
            }
        },
        "container": {
            "job": {
                "reportInterval": 10000
            },
            "taskGroup": {
                "channel": 5
            },
            "trace": {
                "enable": "false"
            }

        },
        "statistics": {
            "collector": {
                "plugin": {
                    "taskClass": "com.alibaba.datax.core.statistics.plugin.task.StdoutPluginCollector",
                    "maxDirtyNumber": 10
                }
            }
        }
    }
}

参考⁠

Tag summary

Content type

Image

Digest

sha256:a5fba2a56…

Size

1.9 GB

Last updated

almost 4 years ago

docker pull sanglr/datax_web:1.0