apache/hive
Apache Hive docker images.
100K+
Apache Hive™ is a popular open-source data warehousing framework that allows users to query large datasets stored in distributed storage systems like Hadoop HDFS, Apache Ozone, Amazon S3, or Microsoft Azure Data Lake Storage (ADLS). It provides a SQL-like (called HiveQL) interface to query, analyze, and manage structured and semi-structured data.
With Hive, users can define tables and schemas, write queries in a familiar SQL-like language and perform various data analysis tasks such as filtering, aggregating, and joining data. Hive supports many popular data formats including CSV, JSON, Avro, ORC, and Parquet.
export HIVE_VERSION=<Hive Tag/Image Version>
Example:
export HIVE_VERSION=4.0.1
This is lightweight and for a quick setup, it uses Derby as metastore db.
docker run -d -p 10000:10000 -p 10002:10002 --env SERVICE_NAME=hiveserver2 --name hive4 apache/hive:${HIVE_VERSION}
For a quick start, launch the Metastore with Derby,
docker run -d -p 9083:9083 --env SERVICE_NAME=metastore --name metastore-standalone apache/hive:${HIVE_VERSION}
docker run -d -p 10000:10000 -p 10002:10002 --env SERVICE_NAME=hiveserver2 \
--env SERVICE_OPTS="-Dhive.metastore.uris=thrift://metastore:9083" \
--env IS_RESUME="true" \
--name hiveserver2-standalone apache/hive:${HIVE_VERSION}
"-Dhive.metastore.uris
is used to specify the external Metastore
docker run -d -p 9083:9083 --env SERVICE_NAME=metastore --env DB_DRIVER=postgres \
--env SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=org.postgresql.Driver -Djavax.jdo.option.ConnectionURL=jdbc:postgresql://postgres:5432/metastore_db -Djavax.jdo.option.ConnectionUserName=hive -Djavax.jdo.option.ConnectionPassword=password" \
--mount source=warehouse,target=/opt/hive/data/warehouse \
--name metastore-standalone apache/hive:${HIVE_VERSION}
Note:
To save the data between container restarts, you can start the HiveServer2 with mounted volume:
docker run -d -p 10000:10000 -p 10002:10002 --env SERVICE_NAME=hiveserver2 \
--env SERVICE_OPTS="-Dhive.metastore.uris=thrift://metastore:9083" \
--mount source=warehouse,target=/opt/hive/data/warehouse \
--env IS_RESUME="true" \
--name hiveserver2 apache/hive:${HIVE_VERSION}
Any other cases your data will not persist!
If you want to use your own core-site.xml
/hdfs-site.xml
/yarn-site.xml
or hive-site.xml
for the service, you can provide the environment variable HIVE_CUSTOM_CONF_DIR
for the command.
For example:
Put the custom configuration file under the directory /opt/hive/conf
and run:
docker run -d -p 9083:9083 --env SERVICE_NAME=metastore \--env DB_DRIVER=postgres -v /opt/hive/conf:/hive_custom_conf --env HIVE_CUSTOM_CONF_DIR=/hive_custom_conf \--name metastore apache/hive:${HIVE_VERSION}
docker exec -it hive4 beeline -u 'jdbc:hive2://localhost:10000/'
You can use beeline as it is installed on your host machine
beeline -u 'jdbc:hive2://localhost:10000/'
Accessed on browser at http://localhost:10002/
show tables;
create table hive_example(a string, b int) partitioned by(c int);
alter table hive_example add partition(c=1);
insert into hive_example partition(c=1) values('a', 1), ('a', 2),('b',3);
select count(distinct a) from hive_example;
select sum(b) from hive_example;
docker pull apache/hive