jenslaufer/rl-trading-preprocessor repository overview

⁠Preprocessing

Create .env file with the properties for preprocessing:

DATA_DIR=C:/Users/jensl/Documents/rl-trader-data
SOURCE=/rldata/reduced
TARGET=/rldata/preprocessed
METHOD=preprocess.iqr.clean
THRESHOLD=3
STEPCSV=
SCALE=

DATA_DIR: is the directory where the data files are located on the host machine, this directorry is mounted to /rldata in docker container

SOURCE: source directory for the files the preprocessing is performed on or path to single file

TARGET: target directory for the processed files

METHOD: method for outlier cleaning: either 'preprocess.iqr.clean' or 'preprocess.zscore.clean'

THRESHOLD: threshold for what is considered as outlier (for zscore in number of standard deviation, for iqr number of interquartile ranges)

STEPCSV: Either '-l' or ''. In case of -l each step during preprocessing is exported to csv

SCALE: Either '-c' or ''. In case of -c the minmax scaling is performed.

To perform preprocessing on host machine you need to execute a docker-compose. This performs the preprocessing in the docker container.

docker-compose up -d --build