A lightweight generic NiFi endpoint to post data into SQL and ElasticSearch databases
968
First we need sql and elastic search databases to insert data into
echo starting mysql
docker run --rm -d -p 3306:3306 --name some-sql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:latest --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
echo starting elasticsearch
docker run --rm -d -p 9200:9200 --name some-es -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.0
Next we need to start the conduit to start sending NiFi POST requests to. Note that the SQL and ES environment variables point to the endpoints:
echo running nifi-sql
docker run --rm -d -eSQL_HOSTS=localhost:3306 -eSQL_USERNAME=root -eSQL_PASSWORD=my-secret-pw -eSQL_DATABASE=nifi -p 8080:8080
A full list of environment variables:
LISTEN : :8080 (default)
SQL_HOSTS : localhost:3306,localhost6:3306 (default)
SQL_DATABASE : nifi (default)
SQL_USERNAME : admin (default)
SQL_PASSWORD : password (default)
SQL_TABLEKEY : type (default)
ES_HOSTS : localhost:9200 (default)
ES_USERNAME : admin (default)
ES_PASSWORD : password (default)
DEBUG : false (default)
Now start posting data!
echo '{"kind":"users","name": "mary","age":23}' > A
echo '{"kind":"users","name": "paul","age":15}' > B
curl -X POST --data "@A" --data "@B" http://localhost:8080/
Content type
Image
Digest
Size
5.4 MB
Last updated
almost 6 years ago
docker pull pschou/nifi-sql