humanoidsctu/romp_bev_trace repository overview

⁠1. Prepare SMPL Model Files

Firstly, please register and download: a. Meta data from this link⁠. Please unzip it, then we get a folder named "smpl_model_data."
b. SMPL model file (SMPL_NEUTRAL.pkl) from "Download version 1.1.0 for Python 2.7 (female/male/neutral, 300 shape PCs)" in official website⁠. Please unzip it and move the SMPL_NEUTRAL.pkl from extracted folder into the "smpl_model_data" folder.
c. Download SMIL model file (DOWNLOAD SMIL) from official website⁠. Please unzip and put it into the "smpl_model_data" folder, so we have "smpl_model_data/smil/smil_web.pkl".
Then we can get a folder in structure like this:

|-- smpl_model_data
|   |-- SMPL_NEUTRAL.pkl
|   |-- J_regressor_extra.npy
|   |-- J_regressor_h36m.npy
|   |-- smpl_kid_template.npy
|   |-- smil
|   |-- |-- smil_web.pkl

⁠2. Docker Build

Dockerfile slightly based on the official Dockerfile from May 2022 https://github.com/Arthur151/ROMP/blob/master/Dockerfile⁠

docker build --rm -t romp_bev_trace .

⁠3. Docker Start-Up

sudo docker run --rm --gpus all -it -e="DISPLAY" -v=/tmp/.X11-unix:/tmp/.X11-unix:rw -v /home/$USER:/home/$USER -v romp_bev_trace /bin/bash

Add -v /media:/media to give access to external hard drives (the hard drive has to be plugged in before the docker is started with this command).

⁠4. Processing commands for ROMP and BEV

⁠Processing with memory profiler

Insert mprof run --include-children --multiprocess --output $output_path/mprofile.dat before the following processing commands

⁠Processing ROMP - single image

romp --mode=image -i $input_path -o $output_path --render_mesh 

⁠Processing ROMP - folder of images or a video

romp --mode=video -i $input_path -o $output_path --render_mesh --save_video

⁠Processing BEV - single image

bev --mode=image -i $input_path -o $output_path --show_items mesh

⁠Processing BEV - folder of images or a video

bev --mode=video -i $input_path -o $output_path --show_items mesh --save_video

⁠Processing TRACE - video

CUDA_VISIBLE_DEVICES=0 trace2 -i $input_path -o $output_path --subject_num=1 --save_video --results_save_dir $output_path

⁠5. Reformating the .npz file to the wanted .csv format (compatible with SMIL)

There are two scripts: The romp_bev_to_csv_multiple_files.py is for processing folder of .npz files, each for single frame. The romp_bev_to_csv_single_file is for processing single .npz file consisting of data of all frames.