Sign inSign up

wangxiang5050/datax

By wangxiang5050

•Updated almost 6 years ago

该镜像包含了datax以及其依赖的java、mvn、python等命令行工具。

Image
0

270

wangxiang5050/datax repository overview

⁠用途

该镜像可以作为datax-executor的基础镜像使用。

⁠v1.0.4-hbase

该版本增加了influxdbreader。

FROM wangxiang5050/datax:v1.0.4
COPY ./hbase11xsqlwriter /datax/plugin/writer/hbase11xsqlwriter

⁠v1.0.4

该版本增加了influxdbreader。

FROM wangxiang5050/datax:v1.0.3
COPY ./influxdbreader /datax/plugin/reader/influxdbreader

⁠v1.0.3

该版本增加了elasticsearch插件。

FROM wangxiang5050/datax:v1.0.2-hbase
COPY ./elasticsearchwriter /datax/plugin/writer/elasticsearchwriter

⁠v1.0.1-hbase

该版本增加了habse插件。另外,替换了/datax/*.py文件,使其适配python3。

⁠Dockerfile
FROM wangxiang5050/datax:v1.0.0
COPY ./hbase20xsqlwriter /datax/plugin/writer/hbase20xsqlwriter
COPY ./hbase20xsqlreader /datax/plugin/reader/hbase20xsqlreader
COPY ./*.py /datax/bin/

⁠v1.0.0 (Deprecated)

该版本镜像包含java 1.8.0_222、mvn 3.6.3、python 3.6.9以及alibaba datax软件包。
datax路径为/datax。
但请不要使用该镜像,因为,pyton3 与/datax/*.py文件不兼容。该问题在v1.0.1-hbase版本中修复。

⁠Dockerfile
### 1. Get Linux
FROM alpine:3.7

### 2. Get Java via the package manager
RUN apk update \
&& apk upgrade \
&& apk add --no-cache bash \
&& apk add --no-cache --virtual=build-dependencies unzip \
&& apk add --no-cache curl \
&& apk add --no-cache openjdk8-jre

### 3. Get Python, PIP

RUN apk add --no-cache python3 \
&& python3 -m ensurepip \
&& pip3 install --upgrade pip setuptools \
&& rm -r /usr/lib/python*/ensurepip && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache

### Get Flask for the app
RUN pip install --trusted-host pypi.python.org flask

####
#### OPTIONAL : 4. SET JAVA_HOME environment variable, uncomment the line below if you need it

ENV JAVA_HOME="/usr/lib/jvm/java-1.8-openjdk"

####

# Downloading and installing Maven
# 1- Define a constant with the version of maven you want to install
ARG MAVEN_VERSION=3.6.3 

# 2- Define a constant with the working directory
ARG USER_HOME_DIR="/root"

# 3- Define the SHA key to validate the maven download
# ARG SHA=b4880fb7a3d81edd190a029440cdf17f308621af68475a4fe976296e71ff4a4b546dd6d8a58aaafba334d309cc11e638c52808a4b0e818fc0fd544226d952544

# 4- Define the URL where maven can be downloaded from
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries

# 5- Create the directories, download maven, validate the download, install it, remove downloaded file and set links
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
  && echo "Downlaoding maven" \
  && curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
  && echo "Unziping maven" \
  && tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
  \
  && echo "Cleaning and setting links" \
  && rm -f /tmp/apache-maven.tar.gz \
  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn

# 6- Define environmental variables required by Maven, like Maven_Home directory and where the maven repo is located

ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
EXPOSE 81    

# install datax
RUN wget http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz -P / \
  && cd / \
  && tar xvf datax.tar.gz \
  && rm -f /datax.tar.gz \
CMD ["date"]

Tag summary

Content type

Image

Digest

Size

1019.1 MB

Last updated

almost 6 years ago

docker pull wangxiang5050/datax:v1.0.4-hbase