kashu98/rlds repository overview

Docker env for downloading Open X-Embodiment Dataset. This Docker runs prepare_open_x.sh⁠ script that downloads and processes the Open X-Embodiment Dataset. The total size of the pre-processed dataset is ~1.2TB.

⁠Usage

docker run -v <DOWNLOAD_DIR>:<DOWNLOAD_DIR> -v <TMP_DIR>:<TMP_DIR> --network host -it kashu98/rlds <DOWNLOAD_DIR> <TMP_DIR>

• <DOWNLOAD_DIR>: directory where source data is temporally stored and the modified data is saved.
• <TMP_DIR>: directory where modified data is temporally stored.

Consider adding --ulimit nofile=200000:200000 and --shm-size=200g if you face any issue.

⁠Using modify_rlds_dataset.py

You can override the entry point by adding --entrypoint /bin/bash in docker run. Then you can run the script with:

/opt/conda/envs/rlds_env/bin/python modify_rlds_dataset.py --dataset=<DATA_NAME> --data_dir=<DATA_DIR> --target_dir=<TARGET_DIR> --mods=<MODS> --n_workers=20 --max_episodes_in_memory=200 

⁠Reference

• Open X-Embodiment⁠
• rlds_dataset_mod⁠