ijerry22/pvrcnn repository overview

⁠Description

This container does not contain the original PV-RCNN, but contains an improved PV-RCNN-based algorithm called Deformable PV-RCNN, also known as DSA PV-RCNN (See https://github.com/AutoVision-cloud/Deformable-PV-RCNN⁠). Deformable PV-RCNN currently ranks first among all open source algorithms in the test results of the KITTI dataset. So my team and I feel that deformable pvrcnn is worth exploring, but after searching, we find that there are few complete tutorials on the installation or use of Deformable PV-RCNN or even PV-RCNN on the Internet in both Chinese and English. the only reference is the author's documentation on GitHub, but the author's document is also very simple and does not have much practical significance, so we are determined to make a docker container. Share it openly with everyone.

⁠Note

This container contains two tag, v1.0 and v2.0-with_opengl. These two tag have different functions, of course, we recommend that you use v2.0-with_opengl. Here are the differences between the two tag:

a. v1.0 is suitable for users who do not need a graphical interface but need to use Deformable PV-RCNN to train the model, because v1.0 is only a nvidia/cuda-based container.

b. v2.0-with_opengl is suitable for users who want to use a graphical interface, because v2.0-with_opengl is developed based on nvidia/cuda-opengl containers. Due to the inclusion of the opengl function, it can support the mayavi library in python for 3D drawing. We recommend you to use this tag because we think Deformable PV-RCNN is an algorithm that requires visual perception, so the support of the graphical interface may make this visual perception more intuitive.

c. The above note is that by default, you use a separate Linux server for deep learning training, while you use a computer to remotely connect to the server to operate. If you are using a Linux computer and use it to train models, you only need pull v1.0, because you can forward the graphical interface to your Linux computer through X11.

d. v1.0 uses Ubuntu 16.04 operating system, built on nvidia/cuda container (See https://hub.docker.com/r/nvidia/cuda⁠), CUDA version is 10.2; v2.0-with_opengl uses Ubuntu 18.04 operating system, built on nvidia/cudagl container (See https://hub.docker.com/r/nvidia/cudagl⁠), CUDA version is 10.2.

⁠Usage

⁠1. Download data from KITTI

First of all, you need to download a few data sets from KITTI's official website (See http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d⁠). You only need to download Left color images of object data set, Right color images of object data set, Velodyne point clouds of object data set, Camera calibration matrices of object data set and Training labels of object data set, but these add up to nearly the size of 50GB, so it may take you a long time to download them.

[Forget to mention, the author of Deformable PV-RCNN also strongly recommends that you download an additional optional package Road plane (You can download it at https://drive.google.com/file/d/1d5mq0RXRnvHPVeKx6Q612z0YRO1t2wAp/view⁠), according to our actual test found that it is actually necessary, if you do not download this package, you may not be able to start training.]

⁠2. Organize KITTI dir. structure

Please organize your file directory structure in the following format:

Deformable-PV-RCNN
├── OpenPCDet
│   ├── data
│   │   ├── kitti
│   │   │   │──ImageSets
│   │   │   │──training
│   │   │   │   ├──calib & velodyne & label_2 & image_2 & (optional: planes)
│   │   │   │──testing
│   │   │   │   ├──calib & velodyne & image_2
│   ├── pcdet
│   ├── tools

Please note: you only need to save the dataset downloaded in the previous step on your server, and you don't have to move the dataset to the docker container, as this will make your container size very large. We recommend that you build the 'kitti' and subfolders in the above directory structure on your server. When you run the docker container, just mount the 'kitti' and its subfolders to the corresponding folder in the docker container.

And then, you should generate the data infos by running the following command:

python -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml

⁠3. Enter the Docker container

If you are using v2.0-with_opengl, you can first obtain the display device ID from the server command line using the following command:

echo $DISPLAY

For example, the ID of the display device we get is 127.0.0.1:10.0. You can enter the docker container with the following command:

docker run -it \
    -v /etc/localtime:/etc/localtime:ro \
    --net=host \
    -e DISPLAY=127.0.0.1:10.0 \
    -v $HOME/slides:/root/slides \
    -v $HOME/.Xauthority:/root/.Xauthority \
    -v /home/ijerry22/workspace/Datasets/KITTI/:/root/Deformable-PV-RCNN/OpenPCDet/data/kitti/ \
    ijerry22/pvrcnn:v2.0-with_opengl /bin/zsh

Briefly introduce the meaning of several important parameters in the above command. The identification of the display device just obtained is written after - e DISPLAY=. The path in front of : in - v / home/ijerry22/workspace/Datasets/KITTI/:/root/Deformable-PV-RCNN/OpenPCDet/data/kitti/ represents the location of your KITTI dataset, and the path after : represents the folder location where the former is mounted in the docker container (You can modify it accordingly according to your actual path). / bin/zsh means to switch to the shell of ZSH. It is important to note that if you do not switch shell, you may not be able to continue running commands in the Docker container.

Finally, don't forget to activate the environment using conda, use the following command:

conda activate pvrcnn

then you can move on to the next step.

⁠4. Getting started

⁠Test and evaluate the pretrained models

cd OpenPCDet/tools

python test.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --ckpt ${CKPT}

python test.py --cfg_file cfgs/kitti_models/def_pv_rcnn.yaml --batch_size 2 --ckpt ${SAVED_CKPT_PATH}/def_pv_rcnn.pth

⁠Train a model

sh scripts/dist_train.sh ${NUM_GPUS} --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --epochs 100

We use --batch_size as 8.

sh scripts/dist_train.sh 8 --cfg_file cfgs/kitti_models/def_pv_rcnn.yaml --batch_size 8 --epochs 100

python train.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --epochs 100

⁠5. Visualize the training results

With regard to the visualization of training results, we use the Mayavi library in Python to achieve. Friends who have known Mayavi should know that its installation process is very difficult and tedious, but it doesn't matter, we have installed it for you here, you can use the Mayavi library directly.

You can find the demo.py program file in the tools folder', and you can forward the visualization results to your computer using the following command:

python demo.py --cfg_file cfgs/kitti_models/pv_rcnn.yaml \
    --data_path ../data/kitti/testing/velodyne/000000.bin \
    --ckpt ../output/kitti_models/def_pv_rcnn/default/ckpt/checkpoint_epoch_100.pth

Briefly introduce the meaning of several parameters of the above command. --cfg_file means to specify the location of the visual configuration file, --data_path means to specify the file or folder location of the fixed-point cloud data, and --ckpt means to specify the location of the trained model.

Result visualization sample (KITTI testing set '000681.bin' file):

⁠ !!! Any questions, you can send an email to jensen.acm(AT)gmail.com.