icra2018/squeezeseg repository overview

⁠SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud

By Bichen Wu, Alvin Wan, Xiangyu Yue, Kurt Keutzer (UC Berkeley)

This repository contains a tensorflow implementation of SqueezeSeg, a convolutional neural network model for LiDAR segmentation. A demonstration of SqueezeSeg can be found below:

Please refer to our video for a high level introduction of this work: https://youtu.be/Xyn5Zd3lm6s⁠. For more details, please refer to our paper: https://arxiv.org/abs/1710.07368⁠. If you find this work useful for your research, please consider citing:

@article{wu2017squeezeseg,
    title={Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud},
    author={Wu, Bichen and Wan, Alvin and Yue, Xiangyu and Keutzer, Kurt},
    journal={ICRA},
    year={2018}
}
@inproceedings{wu2018squeezesegv2,
    title={SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud},
    author={Wu, Bichen and Zhou, Xuanyu and Zhao, Sicheng and Yue, Xiangyu and Keutzer, Kurt},
    booktitle={ICRA},
    year={2019},
}
@inproceedings{yue2018lidar,
    title={A lidar point cloud generator: from a virtual world to autonomous driving},
    author={Yue, Xiangyu and Wu, Bichen and Seshia, Sanjit A and Keutzer, Kurt and Sangiovanni-Vincentelli, Alberto L},
    booktitle={ICMR},
    pages={458--464},
    year={2018},
    organization={ACM}
}

We recently open-sourced the code for SqueezeSegV2, a follow-up work to SqueezeSeg with significantly improved performance. For details, please check out: https://github.com/xuanyuzhou98/SqueezeSegV2⁠

⁠License

SqueezeSeg is released under the BSD license (See LICENSE⁠ for details). The dataset used for training, evaluation, and demostration of SqueezeSeg is modified from KITTI⁠ raw dataset. For your convenience, we provide links to download the converted dataset, which is distrubited under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License⁠.

⁠Installation:

The instructions are tested on Ubuntu 16.04 with python 2.7 and tensorflow 1.0 with GPU support.

• Clone the SqueezeSeg repository:

  git clone https://github.com/BichenWuUCB/SqueezeSeg.git
  

  Copy

  We name the root directory as $SQSG_ROOT.

• Setup virtual environment:

  1. By default we use Python2.7. Create the virtual environment

     virtualenv env
     

     Copy

  2. Activate the virtual environment

     source env/bin/activate
     

     Copy

• Use pip to install required Python packages:

  pip install -r requirements.txt
  

  Copy

⁠Demo:

• To run the demo script:

  cd $SQSG_ROOT/
  python ./src/demo.py
  

  Copy
  If the installation is correct, the detector should write the detection results as well as 2D label maps to $SQSG_ROOT/data/samples_out. Here are examples of the output label map overlaped with the projected LiDAR signal. Green masks indicate clusters corresponding to cars and blue masks indicate cyclists.

  

⁠Training/Validation

• First, download training and validation data (3.9 GB) from this link⁠. This dataset contains LiDAR point-cloud projected to a 2D spherical surface. Refer to our paper for details of the data conversion procedure. This dataset is converted from KITTI⁠ raw dataset and is distrubited under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License⁠.

  cd $SQSG_ROOT/data/
  wget https://www.dropbox.com/s/pnzgcitvppmwfuf/lidar_2d.tgz
  tar -xzvf lidar_2d.tgz
  rm lidar_2d.tgz
  

  Copy

• Now we can start training by

  cd $SQSG_ROOT/
  ./scripts/train.sh -gpu 0 -image_set train -log_dir ./log/
  

  Copy

  Training logs and model checkpoints will be saved in the log directory.

• We can launch evaluation script simutaneously with training

  cd $SQSG_ROOT/
  ./scripts/eval.sh -gpu 1 -image_set val -log_dir ./log/
  

  Copy

• We can monitor the training process using tensorboard.

  tensorboard --logdir=$SQSG_ROOT/log/
  

  Copy

  Tensorboard displays information such as training loss, evaluation accuracy, visualization of detection results in the training process, which are helpful for debugging and tunning models, as shown below:

⁠How to Run with Docker

⁠Linux

⁠Prerequisites

• NVIDIA GPU with CUDA support
• nvidia-docker 2.0⁠

Tested on Ubuntu 16.04.6 with Docker 18.06.1-ce, GPU GeForce 940M, NVIDIA Driver version 410.48.

1. Open a terminal and run the command:

nvidia-docker run --rm -p 8888:8888 icra2018/squeezeseg:latest

1. Run a web browser and open the link: http://localhost:8888/lab/tree/README.ipynb⁠