egpivo/sssd repository overview

⁠SSSD

⁠Prerequisites

• [Local Environment] Install Miniconda⁠
• [Docker] Install Docker⁠

⁠Dataset

1. The NYISO dataset can be downloaded from here⁠.

2. The cleaned data created by the author can be downloaded from this link⁠.

   Note that the cleaned data is created following the scripts in notebooks/dataset_script/nyiso-csv-to-pickle.ipynb and notebooks/dataset_script/nyiso-load-pickle-to-npy.ipynb.

⁠Usage

1. Download data from google drive or S3 to datasets/

   • S3:
     • Enter the AWS credentials
     • Enteraws s3 sync s3://sssd-cp/datasets/ /{repo}/datasets

2. Run the process locally:

   bash scripts/diffusion_process.sh --config {CONFIG_FILE_PATH}
   

   Copy
   • Example: CONFIG_FILE_PATH=configs/config_toy_example.json

3. Run in a container:

   • Adjust CONFIG_FILE in docker-compose.yaml
   • Trigger

     make run-docker
     

     Copy

⁠Suggestions

1. Use CUDA_VISIBLE_DEVICES to specify the number of GPUs. Both training and inference require the same number of GPUs.
2. Use the sample size as the parameter --num_samples in the inference section.
   • Example:

     python sssd/infer.py -c configs/config_SSSDS4-NYISO-3-mix.json --num_samples=128
     

     Copy