The parser can read a given folder with JSON files, filtering and qualifying input data with intent.
339
The parser can read given folder with *.json files, filtering and
qualifying input data with intent & stop words dictionaries and save results
to CSV files by given chunk size.
Minimal dependency on other Go packages, but maximum performance even on large amounts of the input JSON data.
Prepare folder with your data input in the *.json format, create JSON files
with intents and filter.
💡 Note: See the repository's Wiki page to understand structures of JSON files and get general recommendations for preparing the input data.
Next, run container with json2csv parser with options:
docker run --rm -it -v ${PWD}:${PWD} -w ${PWD} koddr/json2csv:latest \
-json /path/to/input/json/folder \
-intents /path/to/intents-file.json \
-filter /path/to/filter-file.json \
-output /path/to/output/csv/folder \
-content-field message \
-min-word-len 5 \
-chunk 1000
Done! 🎉 Output CSV file(s) has a default comma (,) separators between columns.
| Option | Description | Is required? | Default value | Docs link |
|---|---|---|---|---|
-json | set a path to the folder with JSON source file(s) | no | ./json_files | Wiki |
-intents | set a path to the file with intents | no | ./intents.json | Wiki |
-filter | set a path to the file with a filter | no | ./filter.json | Wiki |
-output | set a path to the output folder where the prepared CSV file(s) will be placed | no | ./output_files | Wiki |
-content-field | set a name of the content field (attribute that contains string to qualify and filter) in JSON source file(s) | no | content | |
-min-word-len | set a min word length count to filter input words (if a word is smaller than this option, it will be skipped) | no | 0 | |
-chunk | set a chunk size for one CSV file | no | 5000 |
Imagine that you have many JSON files of the same type in a folder on your computer. The structure of each such file is identical and can be described in a single Go structure.
But your task is not only to turn these files into a structured CSV, but also to make some qualification of incoming data based on dictionaries with intent.
The output should be CSV files (with a specified number of lines in each), where the first column contains the original phrases from JSON files, and the second column will contain the associated intents after qualification.
This is what this Go package solves! ✌️
json2csv is free and open-source software licensed under the
Apache 2.0 License, created and supported with 🩵 for people and
robots by Vic Shóstak.
Content type
Image
Digest
sha256:6c2627a03…
Size
689.1 kB
Last updated
over 3 years ago
docker pull koddr/json2csv