Sign inSign up

rickmyers/dataloader

By rickmyers

•Updated over 7 years ago

Salesforce Dataloader v43.0 on Apline Linux with Microsoft jdbc driver

Image
0

264

rickmyers/dataloader repository overview

⁠Salesforce DataLoader

⁠What is Dataloader?

Dataloader is a Salesforce utility for extracting and importing data into the Salesforce platform. Please refer to the Salesforce Dataloader⁠ for further information.

⁠Image Summary

This docker image is based on Alpine Linux and includes the Microsoft's JDBC driver for Microsoft SQL Server database access. A multi-stage dockerfile is used to create the smallest docker image possible.

Dataloader is executed via the bash shell, the GUI interface is not supported.

⁠Building the base image: dataloader/apline

Note, --build-arg is optional, see Dockerfile for the default version in use.

git clone https:\\github\RickDMyers\dataloader
docker image build --build-arg DATALOADER_VER=43.0 -t dataloader/alpine .
docker tag dataloader/alpine dataloader/alpine:v43.0
docker image prune

⁠Scripts Included

CommandDescriptionUsage
createkey.shCreates an encryption key file.createkey.sh keyfile
encrypt.shUses an encryption key file to encrypt a passwordencrypt.sh password keyfile
decrypt.shUses an encryption key file to decrypt a passworddecrypt.sh password keyfile
process.shExecutes a dataloader process using default config directory.process.sh processName

⁠Directory Structure

DirectoryDescription
binPlace scripts and jar files here. Jars are automatically loaded using the process.sh command
confDefault configuration (used by process.sh)

⁠How to use this image

The best use of the base image is to create your own production image containing all the code and configuration.

Start from the root directory of the application repo and create the following docker file:

⁠Sample Docker File

FROM dataloader/alpine
LABEL maintainer="[email protected]"
LABEL description="your description"

WORKDIR /dataloader

#Copy application specific scripts
COPY ./bin/*.sh ./bin/

RUN mkdir data status log && \
    chmod +x ./bin/*.sh && \
    apk update && \
    apk add nano

COPY ./conf/* ./conf/
COPY ./key/*  ./key/
COPY ./map/*  ./map/

CMD ["sh","/dataloader/bin/dailyload.sh"]

⁠Create load scripts

Create any load scripts and place them in the bin directory. For example, your 'dailyload.sh' script might look like:

TimeStamp=$(date +"%F")_$(date +"%T")
DailyLogFileName="/dataloader/log/dailyload_${TimeStamp}.log"
ProcessLogFilePath="/dataloader/log"


# Execute the Datalaoder Process and log results
processWithLog()
{
    local RunProcessName=$1
    local LogFileName="${ProcessLogFilePath}/Process_${RunProcessName}_${TimeStamp}.Log"
    process.sh ${RunProcessName} > ${LogFileName}
    tail -n 1 ${LogFileName} >> ${DailyLogFileName}
}

echo
echo "************************************"
echo "**** Importing Employee records ****"
echo "****                            ****"
echo

processWithLog EmployeeImportProcess


echo
echo "***********************************************" 
echo "**** Importing Employee Email and Phone's  ****" 
echo "****                                       ****"
echo

processWithLog EmployeeEmailProcess

⁠Create production image

docker image build  -t my/dataloader .

⁠Execute production image

Create a container using the production image.


# Execute the default dailyload.sh
docker run my/dataloader 

# Or run a specific Dataloader process 
docker run my/dataloader sh /dataloader/bin/process.sh {dataloader process name}

⁠Use volume to persist logs

In the sample above, the status, data and log directories are stored inside the container and are available until the container is removed. A volume can be used to store the data longer than the lifespan of the container.

In this example a volume called "dataloader" is created and can be referenced inside the container as /dataloader/local.

docker volume create dataloader

docker run --rm -it  --mount source=dataloader,target=/dataloader/local my/dataloader sh /dataloader/bin/dailyload.sh

⁠Encrypting database connections

Update database-conf.xml to modify the class used to create a datasource.

Change from:

org.apache.commons.dbcp.BasicDataSource to com.salesforce.dataloader.dao.EncryptedDataSource

Use the encrypt.sh tool to encrypt the database password.

For example:

docker --rm -it my/dataloader sh
./bin/encrypt.sh SalesforcePassword+Token ./key/key.txt

Tag summary

Content type

Image

Digest

Size

82.4 MB

Last updated

over 7 years ago

docker pull rickmyers/dataloader