keboola/tde-exporter repository overview

#Description

TDE-exporter exports tables from KBC Storage into TDE file format(Tableau Data Extract⁠).

User can run tde-exporter as any other keboola component or extractor etc., or register its run as orchestration task. After \run finishes, the resulting TDE files will be avaliable in Storage->file uploads section with URL to download.

⁠Tagging

The resulting file is always tagged by default tags tde and table-export, if the job proccess contains runId then the file is also tagged by runId in format runId-<runid>. If the runid is composed(eg 1234.9999) the only so called orchestrator part(before dot) is taken as runId tag value(e.g. from former case it would be: runId-1234). Configuration also allows append of custom tags along with default tags and runid tag(if present), see Configuration->Parameters->Tags.

⁠Empty Values

TDE-exporter handles empty(or null) values as valid TDE specific null values except for string data types where the actual empty value "" is used. No error will ever be raised.

⁠Configuration

TDE-exporter is Keboola Docker component⁠ that supports custom configuration⁠ for \run API call described as follows:

\run API call:

POST https://syrup.keboola.com/docker/tde-exporter/run

request body(config JSON):

{ "configData": { "storage": { "input": { "tables": [ { "source": "in.c-ex-dummy.dummy" } ] } }, "parameters":{ "tags":["sometag"], "typedefs":{ "in.c-ex-dummy.dummy":{ "id":{"type":"number"}, "col1": {"type":"string"} }}}}}

⁠Input tables:

Specified in request body in configData:storage:input:tables as array of {"source":<tableid>}. It can be customized(e.g. only specific columns or values) as described in Keboola Docker Input Mapping⁠

⁠Parameters:

Another part for request body as configData:parameters JSON object, contains:

• tags: array of tags that will be assigned to the resulting file in sapi file uploads.
• typedefs: defines datatypes mapping of source tables columns to destination TDE columns: <table_id(must match source from input tables)>: {<column_name>:{type:<column_type>}...}

e.g: "in.c-ex-dummy.dummy":{ "id":{"type":"number"}, "col1": {"type":"string"} }

##supported column data types 'boolean', 'number', 'decimal','date', 'datetime', 'string'.

⁠Date and Datetime column data types specification

Data for these datatypes can be specified in format described in strptime function⁠. Format can be specified in typedefs column definition e.g. {"col1":{"type":"date", "format":"%m-%d-%Y"}}. If no format is specified then default formats is used:

• date default format: %Y-%m-%d
• datetime default format: %Y-%m-%d %H:%M:%S or %Y-%m-%d %H:%M:%S.%f